How to run the UINT8 DNN face detector example?
I downloaded dkurt's great pre-trained models from the contrib repo following Adrian Rosebrock's blog, and am able to run the floating-point Caffe models, but not the uint8 model which is in Tensorflow format. I'm using a rpi3, but the issue seems unrelated to the architecture: it throws the errors in the model importing stage.
Please help! Solving this would help a lot with the performance on rpi - the caffe models take about 800 millis per frame to process, which is just a little too slow for real-time processing. This must be working for others, as I can see a test being checked in that runs this model...
If I don't use a pbtxt, it stops on unknown nodes, which I believe is expected:
OpenCV(3.4.1) Error: Unspecified error (Unknown layer type Square in op conv4_3_norm/l2_normalize/Square) in populateNet, file /home/pi/opencv/opencv-3.4.1/modules/dnn/src/tensorflow/tf_importer.cpp, line 1582 Traceback (most recent call last): File "detect_faces_video.py", line 33, in <module> net = cv2.dnn.readNetFromTensorflow(model) cv2.error: OpenCV(3.4.1) /home/pi/opencv/opencv-3.4.1/modules/dnn/src/tensorflow/tf_importer.cpp:1582: error: (-2) Unknown layer type Square in op conv4_3_norm/l2_normalize/Square in function populateNet
If I use the pbtxt I found in the repo, then I get this
OpenCV(3.4.1) Error: Assertion failed (layer.input_size() == 1) in populateNet, file /home/pi/opencv/opencv-3.4.1/modules/dnn/src/tensorflow/tf_importer.cpp, line 1485 Traceback (most recent call last): File "detect_faces_video.py", line 32, in <module> net = cv2.dnn.readNetFromTensorflow(model, prototxt) cv2.error: OpenCV(3.4.1) /home/pi/opencv/opencv-3.4.1/modules/dnn/src/tensorflow/tf_importer.cpp:1485: error: (-215) layer.input_size() == 1 in function populateNet
If I export my own pbtxt with this
python tf_text_graph_ssd.py --input opencv_face_detector_uint8.pb --output opencv_face_detector_uint8.pbtxt
then I get the following:
Traceback (most recent call last): File "detect_faces_video.py", line 31, in <module> net = cv2.dnn.readNetFromTensorflow(model, prototxt) cv2.error: OpenCV(3.4.1) /home/pi/opencv/opencv-3.4.1/modules/dnn/src/tensorflow/tf_importer.cpp:553: error: (-2) Input layer not found: BoxPredictor_0/ClassPredictor/BiasAdd in function connect
import cv2
print("[INFO] loading model...")
#prototxt = "deploy.prototxt.txt"
#model = "res10_300x300_ssd_iter_140000.caffemodel"
#model = "res10_300x300_ssd_iter_140000_fp16.caffemodel"
#net = cv2.dnn.readNetFromCaffe(prototxt, model) # This works with either model
prototxt = "opencv_face_detector_uint8.pbtxt"
model = "opencv_face_detector_uint8.pb"
net = cv2.dnn.readNetFromTensorflow(model, prototxt) # This doesn't
@BViktor, please try to import model again with a newer state of OpenCV. Actually, despite weights are in UINT8 they are converted to FP32 because only FP32 computations are supported on CPU for now. However you can achieve better efficiency by varying input image size. Model has been trained on 300x300 images but it also works well on lower resolutions. In example, one of tutorials resizes inputs to 128x96. Note that the lower image the less accurate predictions could be. So you can use an origin caffemodel without OpenCV recompilation.
Wow, this is amazing! I played around with the settings and managed to get pretty reasonable detection with 80x80 inputs at 10 FPS on the Raspberry Pi 3! I'm using a low confidence threshold (0.15), but I guess due to the low resolution it doesn't seem to have many false positives. At 128x128 speed is about 3.5 FPS, but interestingly at 96x96 or 88x88 the speed is still the same. We hit some sweet spot at 80x80 that the speed increases drastically. I wonder why? Also very interestingly, the 80x80 case uses two CPU cores, whereas the 88, 96 etc. only uses one.
@BViktor, good luck! You may also keep an origin aspect ratio so height could be even less than 80.