read nets with "Caffe python layer" in OpenCV

asked 2019-03-12 03:11:59 -0500

updated 2019-03-12 06:20:52 -0500

berak gravatar image

There is a caffe model (here) which has a "caffe python layer". I can read it normally in python using caffe.Net(...) command (caffe already compiled with WITH_PYTHON_LAYER=1). However, due to existing "caffe python layer", I can't read it using cv2.dnn.readNet in OpenCV4. I mean:

net = caffe.Net(model_arch_path, model_weight_path, caffe.TEST) #caffe method

works fine, but below one does not (It generates errors about that existing "cafe python layer":

net = cv2.dnn.readNet(model_arch_path, model_weight_path, caffe.TEST) #OpenCV4 method

This is a general question, not specific to this network. How should I read caffe models which include "python-layer" by OpenCV? if not directly readable in OpenCV, is there any easy way to first read it by caffe read net method, and then cast it to cv2.dnn.readNet type?

I've already easily read and work original Yolov3-darknet with OpenCV. But due to some reasons I want to use it's caffe conversion. The reason is I want to use Yolov3 on some AI accelerator USB sticks (like intel NCS2, Orange pi AI stick,...). They work well with caffe models, but does not support or does not work without issue with original darknet model.

edit retag flag offensive close merge delete

Comments

sorry, but this is far to unspecific, now. we need details, details, details to help you.

which network / model, exactly ? link ? are there any errors to show us ? which attributes, exactly ? can you show us the prototxt ? what is the custom python layer about ? and so on ...

please edit your question, and try to make it more specific, thank you.

berak gravatar imageberak ( 2019-03-12 03:30:49 -0500 )edit

hello, I edit it. sorry for unclearance. I tried to make it more clear

hamzeah gravatar imagehamzeah ( 2019-03-12 03:43:09 -0500 )edit

just curious, is there any reason to use a pytorch version of yolov3, implemented with caffe ?

(sounds like a major detour to me).

can't you use the original yolo model (which can be easily read from opencv's dnn) ?

apart from that, here's a python example on how to implement custom network layers

and still, not enough information in your question, to really help you.

berak gravatar imageberak ( 2019-03-12 03:48:15 -0500 )edit
1

I've already easily read and work original Yolov3-darknet with OpenCV. But due to some reasons I want to use it's caffe conversion. The reason is I want to use Yolov3 on some AI accelerator USB sticks (like intel NCS2, Orange pi AI stick,...). They work well with caffe models, but does not support or does not work without issue with original darknet model. ( I add this to original post)

hamzeah gravatar imagehamzeah ( 2019-03-12 03:58:57 -0500 )edit

^^ ah, sure, makes more sense now, thanks for the edit!

berak gravatar imageberak ( 2019-03-12 04:05:15 -0500 )edit

c++, but an example of custom bilinear up/downsampling , but again:

we can only help you with opencv's dnn models here, not with caffe.net

berak gravatar imageberak ( 2019-03-12 04:16:06 -0500 )edit

OpenCV can read caffe models with ReadNet, or ReadNetFromCaffe. But when there is a "Caffe python layer" in the model, It gives error pointing to that specific type of layer. From your statement, it seems handling such type of layer is not supported in OpenCV.

hamzeah gravatar imagehamzeah ( 2019-03-12 05:16:01 -0500 )edit

imho, you have to rewrite it to be a custom dnn layer (reshape -> getMemoryShapes, etc) , and register it with cv2.dnn.

(afaik, it's meant to be a replacement for "python" (or other custom) layers, but again, you have to adapt it to dnn needs from caffe)

berak gravatar imageberak ( 2019-03-12 05:26:30 -0500 )edit

Oh, I see. thank you very much. I will try to do that.

hamzeah gravatar imagehamzeah ( 2019-03-12 05:34:57 -0500 )edit