Ask Your Question
0

OpenCV on Android: net.forward yields "blobs.size() >= 2" Assertion failed

asked 2019-08-09 04:29:59 -0600

4ndre gravatar image

Following this tutorial from openCV, and it should be straight forward. However, it crashes with an assertion fail on the net.forward, that I cannot resolve/find anywhere else.

Thought this problem seemed similar and tried to go through the fix/problem finding. However, restarting the discussion and trials showed it is likely not the same. I used initially 3.4.3, which did not support the same Mat type somehow. Updated to 3.4.7 now, and can confirm the blob size is okay (generated from image). Tried also various other prototxt and caffemodels, but doubt by now that the problem lies there (works if the files are okay, otherwise the net loading fails). The key code should be this:

// Load a network.
public void onCameraViewStarted(int width, int height) {
    String proto = getPath("deploy.prototxt", this);
    String weights = getPath("MobileNetSSD_deploy.caffemodel", this);
    net = Dnn.readNetFromCaffe(proto, weights);
    Log.i(TAG, "Network loaded successfully");
}
public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
    // Get a new frame
    Mat frame = inputFrame.rgba();
    Imgproc.cvtColor(frame, frame, Imgproc.COLOR_RGBA2RGB);
    // Forward image through network.
    Mat blob = Dnn.blobFromImage(frame, 0.007843,
            new Size(300, 300),
            new Scalar(127.5, 127.5, 127.5));

    net.setInput(blob);
    Mat detections = net.forward(); //***215 ASSERTION FAILED occurs***
    int cols = frame.cols();
    int rows = frame.rows();
    detections = detections.reshape(1, (int)detections.total() / 7);
    for (int i = 0; i < detections.rows(); ++i) {
        double confidence = detections.get(i, 2)[0];
        if (confidence > 0.2) {
            int classId = (int)detections.get(i, 1)[0];
            int left   = (int)(detections.get(i, 3)[0] * cols);
            int top    = (int)(detections.get(i, 4)[0] * rows);
            int right  = (int)(detections.get(i, 5)[0] * cols);
            int bottom = (int)(detections.get(i, 6)[0] * rows);
            // Draw rectangle around detected object.
            Imgproc.rectangle(frame, new Point(left, top), new Point(right, bottom),
                    new Scalar(0, 255, 0));
            String label = classNames[classId] + ": " + confidence;
            int[] baseLine = new int[1];
            Size labelSize = Imgproc.getTextSize(label, Core.FONT_HERSHEY_SIMPLEX, 0.5, 1, baseLine);
            // Draw background for label.
            Imgproc.rectangle(frame, new Point(left, top - labelSize.height),
                    new Point(left + labelSize.width, top + baseLine[0]),
                    new Scalar(255, 255, 255), Core.FILLED);
            // Write class name and confidence.
            Imgproc.putText(frame, label, new Point(left, top),
                    Core.FONT_HERSHEY_SIMPLEX, 0.5, new Scalar(0, 0, 0));
        }
    }
    return frame;
}
public void onCameraViewStopped() {}
// Upload file to storage and return a path.
private static String getPath(String file, Context context) {
    AssetManager assetManager = context.getAssets();
    BufferedInputStream inputStream = null;
    try {
        // Read data from assets.
        inputStream = new BufferedInputStream(assetManager.open(file));
        byte[] data = new byte[inputStream.available()];
        inputStream.read(data);
        inputStream.close();
        // Create copy file in storage.
        File outFile = new File(context.getFilesDir(), file);
        FileOutputStream os = new FileOutputStream(outFile);
        os.write(data);
        os.close();
        // Return a path to file which may be read in common way.
        return outFile.getAbsolutePath();
    } catch (IOException ex) {
        Log.i(TAG, "Failed to upload a file");
    }
    return "";
}

The full error message is

cv::Exception: OpenCV(3.4.7) /build/3_4_pack-android/opencv/modules/dnn/src/layers/batch_norm_layer.cpp:39 ...

(more)
edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
0

answered 2019-08-14 04:36:49 -0600

4ndre gravatar image

After some days of research in different directions, I found the problem: the frame format should be BGR, not RGB! That means

Imgproc.cvtColor(frame, frame, Imgproc.COLOR_RGBA2BGR);
edit flag offensive delete link more

Question Tools

1 follower

Stats

Asked: 2019-08-09 04:29:59 -0600

Seen: 354 times

Last updated: Aug 09 '19