Process to run OpenCV Inference from TF model
I need to do the inference of a Tensorflow model in C++ OpenCV 4.0.1.
My model is a custom proprietary (I can't past code here) Faster-RCNN model. It's a huge model develop from scratch, by still do object detection and is based on Faster-RCNN architecture. Not very recent, it's develop with Tensorflow 1.8.
I found several documentation about loading a TF model with OpenCV DNN module:
https://github.com/opencv/opencv/wiki... says that "The provided API (for C++ and Python) is very easy to use, just load the network and run it. Multiple inputs/outputs are supported"
https://github.com/opencv/opencv/wiki... show how to do it.
I first tried to load and run a model from the documentation (Faster-RCNN ResNet-50) and it works well with the corresponding .pbtxt file. So there is no problem with my OpenCV install or my C++ code (That is inspired from this)
So with my own model, first problem appear: I have only .pb file. The code used for training don't generate .pbtxt, and this is not an option for the moment to modify it. From the OpenCV documentation, the readNetFromTensorflow() function can load a model without the .pbtxt file, this is an optional parameter. It seems that the .pbtxt file is only to help OpenCV to load the graph. With only my .pb, I get this error:
cpp
terminate called after throwing an instance of 'cv::Exception'
what(): OpenCV(4.0.1) /opencv-4.0.1/modules/dnn/src/tensorflow/tf_importer.cpp:1377: error: (-215:Assertion failed) scaleMat.type() == CV_32FC1 in function 'populateNet'
I found this error here with no answer.
So maybe the solution is to create a .pbtxt. The documentation give a script to generate it for Faster-RCNN model, but need a configuration file. It exactly says "Pass a configuration file which was used for training to help script determine hyper-parameters." I have no such thing due to the fact that it's not a model from TF OD API model zoo, but a custom model created from scratch. Should I create a config file like these to use the script and generate a .pbtxt for my model ?
TL;DR: it's not specified in the documentation if the dnn module can only load model from TF OD API model zoo, like always in the examples. Indeed, it says that "You can build your own model", but the documentation that follow doesn't correspond. The script generating the .pbtxt need a config file that is a thing from TF OD API model zoo.
Does anyone can explain me what I don't understand ?
There is no TensorFlow model zoo. There is TensorFlow Object Detection API Model Zoo which provides training scripts to train object detection models. From wiki:
This wiki describes how to work with object detection models trained using TensorFlow Object Detection API. OpenCV 3.4.1 or higher is required.. If you model trained not with TF OD API you need a different approach to enable it.Thank you for your response. Sorry to not to express myself clearly.
So this documentation does not concern me if I understand well. So the only way I have to use OpenCV with my model is to find where the error come from ? Do you have an idea about the solution ? Do you know if the .pbtxt is really needed, and there is particular step to follow to create one that will work with my model ?
@ypetit, It depends on the way you defined the graph. As documentation says, TensorFlow uses low level operators to express deep learning layers. In example, SSDs from Caffe have 7 layers in SSD head (6
PriorBoxlayers and oneDetectionOutput) excluding flatten layers and others. However TensorFlow implement it by about 3000 nodes suchSum,Muletc. We use supportive scripts to cut them but it works only for models from TensorFlow Object Detection API because we know how they are designed.