Dnn.forward crash in Android ndk [closed]

asked 2018-12-27 11:34:14 -0500

timtsoitt gravatar image

Hi,

I am using OpenCV for Android 4.01. I am trying to use caffe dnn for face detection. I construct a Mat image for the captured image. Then I use haar-based face detection for detecting frontal face and use dnn-based face detection to crop the face contour precisely.

The problem is the application crashes when the application try to execute net.forward(), below is error message:

E/cv::error(): OpenCV(4.0.1) Error: Assertion failed (inputs[0].size[1] % blobs[0].size[1] == 0) in forward, file /build/master_pack-android/opencv/modules/dnn/src/layers/convolution_layer.cpp, line 1119

However the mat data has passed haar-based face detection before it enter dnn-based face detection. I am quite sure haar-based function has no problem.

This is dnn function code:

bool dnnFace(const Mat img, Rect *faceRoi) {

cv::Mat inputBlob = cv::dnn::blobFromImage(img, 0.1, Size(224, 224), mean(img), false, false);

net.setInput(inputBlob);

cv::Mat detection = net.forward();
cv::Mat detectionMat(detection.size[2], detection.size[3], CV_32F, detection.ptr<float>());

int height = img.rows;
int width = img.cols;

for (int i = 0; i < detectionMat.rows; i++) {
    float confidence = detectionMat.at<float>(i, 2);

    if (confidence > 0.9) {
        int x1 = static_cast<int>(detectionMat.at<float>(i, 3) * width);
        int y1 = static_cast<int>(detectionMat.at<float>(i, 4) * height);
        int x2 = static_cast<int>(detectionMat.at<float>(i, 5) * width);
        int y2 = static_cast<int>(detectionMat.at<float>(i, 6) * height);

        *faceRoi = Rect(cv::Point(x1, y1), cv::Point(x2, y2));
        return true;
    }
}
return false;

}

This is haar-based function:

bool findFace(const Mat img, Rect *faceRoi) {

std::vector<Rect> faces;
Mat gray, smallImg;

cvtColor(img, gray, COLOR_BGR2GRAY); // Convert to Gray Scale

// Resize the Grayscale Image
resize(gray, smallImg, Size(), 1, 1, INTER_LINEAR);
equalizeHist(smallImg, smallImg);

faceCascade.detectMultiScale(smallImg, faces, 1.1, 2, 0 | CASCADE_SCALE_IMAGE, Size(75, 75));

for (size_t i = 0; i < faces.size(); i++) {
    *faceRoi = faces[i];
    Mat smallImgROI;
    std::vector<Rect> eyeObjects;

    smallImgROI = smallImg(*faceRoi);
    eyeCascade.detectMultiScale(smallImgROI, eyeObjects, 1.1, 6, 0 | CASCADE_SCALE_IMAGE,
                                Size(30, 30));

    if (2 == eyeObjects.size()) {
        return true;
    }
}

return false;

}

I have also tested these codes in Visual Studio 2017 with OpenCV 4.01 and there is no crash. Anyone has idea? Thanks.

edit retag flag offensive reopen merge delete

Closed for the following reason the question is answered, right answer was accepted by timtsoitt
close date 2018-12-27 13:16:57.769058

Comments

1

the haar method expects grayscale images, the cnn one bgr ones for input (you seem to feed it grayscale)

can you check that again ?

 inputs[0].size[1] % blobs[0].size[1] == 0

if that failed, something is wrong with your channels

berak gravatar imageberak ( 2018-12-27 11:41:45 -0500 )edit
1

also: scale should be 1.0, and mean (104, 177, 123)

(you need to use the mean of the train db,not the one of your image here)

berak gravatar imageberak ( 2018-12-27 11:51:37 -0500 )edit
1

@berak, As for me Error: Assertion failed (inputs[0].size[1] % blobs[0].size[1] == 0) in forward is one of the most regular assertion we face in questions. And this is just because of wrong number of input channels (i.e. BGRA instead of BGR or something like this). I'd like to prepare a PR with some human-readable message for this error.

dkurt gravatar imagedkurt ( 2018-12-27 13:00:51 -0500 )edit

I find that my input Mat is in RGB, making conversion from RGB to BGR solve the problem. Thanks for the hint from @berak. And also thanks for pointing out my code issue.

cv::cvtColor(img, bgr, COLOR_RGB2BGR);

timtsoitt gravatar imagetimtsoitt ( 2018-12-27 13:16:04 -0500 )edit

ah, right BGRA won't fit either ;)

@timtsoitt , please check img.channels() . if that's 4 (default on android !), you need a:

cvtColor(img, img, COLOR_BGRA2BGR);

before feeding it into blobFromImage()

berak gravatar imageberak ( 2018-12-27 13:21:05 -0500 )edit
1

@berak, oh it is 4 channels. Thanks again :) Also credit to @dkurt ! (Sorry for missing your name in above comment)

timtsoitt gravatar imagetimtsoitt ( 2018-12-27 13:27:10 -0500 )edit

@timtsoitt, Never mind. Glad we helped you.

dkurt gravatar imagedkurt ( 2018-12-27 14:05:24 -0500 )edit
1

I'd like to prepare a PR with some human-readable message for this error.

yes, good idea. !

all i could come up with so far is:

CV_Check(inputs[0].size[1] % blobs[0].size[1], 0, "unexpected input channel size");

which results in:

terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(4.0.0-pre) src/cv.cpp:74: error: (-2:Unspecified error) in function 'int main()'
> unexpected input channel size:
>     '0'
> where
>     'inputs[0].size[1] % blobs[0].size[1]' is 2

(better, but still quite ugly...)

berak gravatar imageberak ( 2018-12-29 07:08:48 -0500 )edit