Dnn.forward crash in Android ndk [closed]
Hi,
I am using OpenCV for Android 4.01. I am trying to use caffe dnn for face detection. I construct a Mat image for the captured image. Then I use haar-based face detection for detecting frontal face and use dnn-based face detection to crop the face contour precisely.
The problem is the application crashes when the application try to execute net.forward(), below is error message:
E/cv::error(): OpenCV(4.0.1) Error: Assertion failed (inputs[0].size[1] % blobs[0].size[1] == 0) in forward, file /build/master_pack-android/opencv/modules/dnn/src/layers/convolution_layer.cpp, line 1119
However the mat data has passed haar-based face detection before it enter dnn-based face detection. I am quite sure haar-based function has no problem.
This is dnn function code:
bool dnnFace(const Mat img, Rect *faceRoi) {
cv::Mat inputBlob = cv::dnn::blobFromImage(img, 0.1, Size(224, 224), mean(img), false, false);
net.setInput(inputBlob);
cv::Mat detection = net.forward();
cv::Mat detectionMat(detection.size[2], detection.size[3], CV_32F, detection.ptr<float>());
int height = img.rows;
int width = img.cols;
for (int i = 0; i < detectionMat.rows; i++) {
float confidence = detectionMat.at<float>(i, 2);
if (confidence > 0.9) {
int x1 = static_cast<int>(detectionMat.at<float>(i, 3) * width);
int y1 = static_cast<int>(detectionMat.at<float>(i, 4) * height);
int x2 = static_cast<int>(detectionMat.at<float>(i, 5) * width);
int y2 = static_cast<int>(detectionMat.at<float>(i, 6) * height);
*faceRoi = Rect(cv::Point(x1, y1), cv::Point(x2, y2));
return true;
}
}
return false;
}
This is haar-based function:
bool findFace(const Mat img, Rect *faceRoi) {
std::vector<Rect> faces;
Mat gray, smallImg;
cvtColor(img, gray, COLOR_BGR2GRAY); // Convert to Gray Scale
// Resize the Grayscale Image
resize(gray, smallImg, Size(), 1, 1, INTER_LINEAR);
equalizeHist(smallImg, smallImg);
faceCascade.detectMultiScale(smallImg, faces, 1.1, 2, 0 | CASCADE_SCALE_IMAGE, Size(75, 75));
for (size_t i = 0; i < faces.size(); i++) {
*faceRoi = faces[i];
Mat smallImgROI;
std::vector<Rect> eyeObjects;
smallImgROI = smallImg(*faceRoi);
eyeCascade.detectMultiScale(smallImgROI, eyeObjects, 1.1, 6, 0 | CASCADE_SCALE_IMAGE,
Size(30, 30));
if (2 == eyeObjects.size()) {
return true;
}
}
return false;
}
I have also tested these codes in Visual Studio 2017 with OpenCV 4.01 and there is no crash. Anyone has idea? Thanks.
the haar method expects grayscale images, the cnn one bgr ones for input (you seem to feed it grayscale)
can you check that again ?
if that failed, something is wrong with your channels
also: scale should be 1.0, and mean (104, 177, 123)
(you need to use the mean of the train db,not the one of your image here)
@berak, As for me
Error: Assertion failed (inputs[0].size[1] % blobs[0].size[1] == 0) in forward
is one of the most regular assertion we face in questions. And this is just because of wrong number of input channels (i.e. BGRA instead of BGR or something like this). I'd like to prepare a PR with some human-readable message for this error.I find that my input Mat is in RGB, making conversion from RGB to BGR solve the problem. Thanks for the hint from @berak. And also thanks for pointing out my code issue.
cv::cvtColor(img, bgr, COLOR_RGB2BGR);
ah, right BGRA won't fit either ;)
@timtsoitt , please check
img.channels()
. if that's 4 (default on android !), you need a:before feeding it into blobFromImage()
@berak, oh it is 4 channels. Thanks again :) Also credit to @dkurt ! (Sorry for missing your name in above comment)
@timtsoitt, Never mind. Glad we helped you.
yes, good idea. !
all i could come up with so far is:
which results in:
(better, but still quite ugly...)