Android OpenCV poor performance with JNI
<p>I have a huge problem with OpenCV 3.10 under Android. I am developing an App which does TemplateMatching of an Camera Preview.
The first approach has been to use the OpenCV Java Wrapper which worked okay. One Processing cycle took about 3.6s. To speed this up i redeveloped the code in C++. For some reason since of that the execution of one cycle started to take up to 35s.
Trying to speed this up and leverage the multithreading abilities i move the JNI execution to an AsyncTask. Since that, a single execution takes up to 65s.</p>

<p>I am using the gradle experimental plugin 0.7.0 which is considered stable and the most recent NDK (12.1 as of now).</p>

<p>here's my module build.gradle</p>

<pre><code>    ndk {
        moduleName "OpenCVWrapper"
        ldLibs.addAll(["android", "log", "z"])
        cppFlags.add("-std=c++11")
        cppFlags.add("-fexceptions")
        cppFlags.add("-I"+file("src/main/jni").absolutePath)
        cppFlags.add("-I"+file("src/main/jni/opencv2").absolutePath)
        cppFlags.add("-I"+file("src/main/jni/opencv").absolutePath)
        stl = "gnustl_shared"
        debuggable = "true"
    }
    productFlavors {
        create("arm") {
            ndk.with {
                abiFilters.add("armeabi")
                String libsDir = file('../openCVLibrary310/src/main/jniLibs/armeabi/').absolutePath+'/'
                ldLibs.add(libsDir + "libopencv_core.a")
                ldLibs.add(libsDir + "libopencv_highgui.a")
                ldLibs.add(libsDir + "libopencv_imgproc.a")
                ldLibs.add(libsDir + "libopencv_java3.so")
                ldLibs.add(libsDir + "libopencv_ml.a")

            }
        }
        create("armv7") {
            ndk.with {
                abiFilters.add("armeabi-v7a")
                String libsDir = file('../openCVLibrary310/src/main/jniLibs/armeabi-v7a/').absolutePath+'/'
                ldLibs.add(libsDir + "libopencv_core.a")
                [... and so on ...]
</code></pre>

<p>So heres the Android-Java code which executed in about 3-4 seconds:</p>

<pre><code>    // data is byte[] from camera
    Mat yuv = new Mat(height+height/2, width, CvType.CV_8UC1);
    yuv.put(0,0,data);
    Mat input = new Mat(height, width, CvType.CV_8UC3);

    Imgproc.cvtColor(yuv, input, Imgproc.COLOR_YUV2RGB_NV12, 3);
    yuv.release();

    int midPoint = Math.min(input.cols(), input.rows())/2;
    Mat rotated = new Mat();
    Imgproc.warpAffine(input, rotated,
            Imgproc.getRotationMatrix2D(new Point(midPoint, midPoint), 270, 1.0),
            new Size(input.rows(), input.cols()));
    input.release();

    android.util.Size packageRect = midRect.getSize();
    input.release();

    Rect r = new Rect(((rotated.cols()/2)-(packageRect.getWidth()/2)),
            ((rotated.rows()/2)-(packageRect.getHeight()/2)),
            packageRect.getWidth(), packageRect.getHeight());
    Mat cut = new Mat(rotated, r);
    Mat scaled = new Mat();
    Imgproc.resize(cut,scaled, new Size(323, 339), 0, 0, Imgproc.INTER_AREA);
    Imgcodecs.imwrite(getExternalFileName("cutout").getAbsolutePath(), cut);
    cut.release();

    Mat output = new Mat();
    Imgproc.matchTemplate(pattern, scaled, output, Imgproc.TM_CCOEFF_NORMED);
    Core.MinMaxLocResult tmplResult = Core.minMaxLoc(output);

    findPackage(tmplResult.maxLoc.x+150);
    scaled.release();
    input.release();
    output.release();
    cut.release();
</code></pre>

<p>In turn thats the C++ code to do exactly the same:</p>

<pre><code>JNIEXPORT void JNICALL Java_at_identum_planogramscanner_ScanActivity_scanPackage(JNIEnv *env, jobject instance, jbyteArray input_, jobject data, jlong output, jint width, jint height, jint rectWidth, jint rectHeight) {
jbyte *input = env-&gt;GetByteArrayElements(input_, NULL);

jclass resultDataClass = env-&gt;GetObjectClass(data);
jmethodID setResultMaxXPos = env-&gt;GetMethodID(resultDataClass, "setMaxXPos", "(I)V");
jmethodID setResultMinXPos = env-&gt;GetMethodID(resultDataClass, "setMinXPos", "(I)V");
jmethodID setResultMinVal = env-&gt;GetMethodID(resultDataClass, "setMinVal", "(F)V");
jmethodID setResultMaxVal = env-&gt;GetMethodID(resultDataClass, "setMaxVal", "(F)V");

LOGE("Before work");

Mat convert(height+height/2, width, CV_8UC1, (unsigned char*)input);
Mat img(height, width, CV_8UC3);
cvtColor(convert, img, CV_YUV2RGB_NV12, 3);
convert.release();

LOGE("After Colorconvert");

int midCoord = min(img.cols, img.rows)/2;
Mat rot;
Mat rotMat = getRotationMatrix2D(Point2f(midCoord,midCoord), 270, 1.0);
warpAffine(img, rot, rotMat, Size(img.rows, img.cols));
rotMat.release();

LOGE("After Rotation");

Rect r(
        (rot.cols/2-rectWidth/2),
        (rot.rows/2-rectHeight/2),
        rectWidth, rectHeight );
Mat cut(rot,r);
rot.release();

LOGE("After Cutting");

Mat scaled(Size(323, 339), CV_8UC3);
resize(cut, scaled, Size(323,339),0,0,INTER_AREA);
cut.release();

LOGE("After Scaling");

Mat match(pattern.cols, 1, CV_8UC1);
matchTemplate(pattern, scaled, match, TM_SQDIFF_NORMED);
scaled.release();

LOGE("After Templatematching and normalize");

double minVal; double maxVal; Point minLoc; Point maxLoc;
minMaxLoc(match, &amp;minVal, &amp;maxVal, &amp;minLoc, &amp;maxLoc, Mat());

img.release();
env-&gt;CallVoidMethod(data, setResultMinXPos, minLoc.x);
env-&gt;CallVoidMethod(data, setResultMaxXPos, maxLoc.x);
env-&gt;CallVoidMethod(data, setResultMinVal, minVal);
env-&gt;CallVoidMethod(data, setResultMaxVal, maxVal);

LOGE("After Calling JNI funcs");

env-&gt;ReleaseByteArrayElements(input_, input, 0);
</code></pre>

<p>as you can see it is practically exactly the same work and i expected it to run a little faster than written in Android-Java but for sure not 10 times slower and definetely not 20 times slower when ran from AsyncTask.</p>

<p>My best conclusion is that the .a archives of OpenCV need some kind of Compiler settings to speed up as much as possible. I hope anyone can point me into the right direction!</p>

<p>Thanks in advance!</p>