Ask Your Question
0

Assertion Error with Custom Trained Caffe

asked 2018-04-20 08:44:10 -0600

Mary-Ann gravatar image

updated 2018-04-24 11:09:12 -0600

Hi,

I would like to use the OpenCV DNN framework with a custom trained Caffe network. I am using the newest OpenCV Version and contrib libs, I compiled two weeks ago, I guess. I have already checked the dnn samples, and with the precompiled SqueezeDet and SqueezeNet examples everything works just fine. But if I load my network I get the following exception:

OpenCV(3.4.1-dev) Error: Unspecified error (Can't create layer "cluster" of type "Python") in getLayerInstance, file opencv-master/modules/dnn/src/dnn.cpp, line 388 terminate called after throwing an instance of 'cv::Exception' what(): OpenCV(3.4.1-dev) opencv-master/modules/dnn/src/dnn.cpp:388: error: (-2) Unspecified error: Can't create layer "cluster" of type "Python" in function getLayerInstance

The network should be fine, I have tested it outside the OpenCV framework. The caffe version is 0.15.14, digits version is 5.1-dev. I use OpenCV in C++ not in Python, could it be the error?

Thanks a lot for any answers! Mary

EDIT: Here is the last layer of the prototxt, that probably causes the problem:

layer {    
  name: "cluster"    
  type: "Python"    
  bottom: "coverage"    
  bottom: "bboxes"    
  top: "bbox-list"    
  python_param {    
    module: "caffe.layers.detectnet.clustering"    
    layer: "ClusterDetections"    
    param_str: "1248, 352, 16, 0.6, 3, 0.02, 22, 1"    
  }    
}
edit retag flag offensive close merge delete

Comments

2

no, opencv does not have a "cluster" layer , so far.

berak gravatar imageberak ( 2018-04-20 08:54:56 -0600 )edit

@Mary-Ann, Are there any references to this layer? I guess "cluster" is only the name of layer but not it's type.

dkurt gravatar imagedkurt ( 2018-04-20 09:00:02 -0600 )edit

Hi, I'm not sure that I get your question right, but here is how I use my network in my application: ReadNetFromCaffe(prototxt, caffemodel) SetInput(inputBlob) forward() I don't use any arguments in the method forward() I only want that it propagates through the whole network. It works with squeezeNet and squeezeDet in this way, and also with specified layers, but not with my network...

Mary-Ann gravatar imageMary-Ann ( 2018-04-20 11:56:32 -0600 )edit
2

@Mary-Ann , please add (at least) the prototxt to the question

berak gravatar imageberak ( 2018-04-20 11:59:04 -0600 )edit
1

Hi, I updated the question, I hope it helps to solve my problem

Mary-Ann gravatar imageMary-Ann ( 2018-04-24 08:32:27 -0600 )edit
1

@Mary-Ann - wow, that broke the editor of this webpage ;)

maybe we need ONLY the last layer, and the resp. python code for that. then we can see, if there's any ready-made opencv replacement code for this

berak gravatar imageberak ( 2018-04-24 10:09:42 -0600 )edit

I updated the question so that only the last layer can be seen. The network was trained using DIGITS (https://github.com/NVIDIA/DIGITS/tree...), with a dataset similar to KITTI (http://www.cvlibs.net/datasets/kitti/...). Just like in the example DetectNet was used. I have just realized, that I dont use the binaryproto of the model. I am currently working on the reading in of the file. But I dont think so, that the wrong mean values cause the actual problem.

Mary-Ann gravatar imageMary-Ann ( 2018-04-27 09:31:57 -0600 )edit
1

@Mary-Ann, dkurt is already working on support for custom python layers, watch this pr , and update, once it's merged ;)

berak gravatar imageberak ( 2018-04-27 09:40:26 -0600 )edit

@berak, yeah, I can handle to test it with an implementation from https://github.com/NVIDIA/caffe/blob/....

dkurt gravatar imagedkurt ( 2018-04-27 10:16:31 -0600 )edit

2 answers

Sort by ยป oldest newest most voted
0

answered 2018-05-03 04:11:24 -0600

dkurt gravatar image

updated 2018-05-03 04:17:28 -0600

Way to reproduce

Get https://github.com/NVIDIA/caffe/blob/... and comment all the layers which have phase: TRAIN or stage: "val". Try to import using OpenCV (at least https://github.com/opencv/opencv/comm...).

Error message

Can't create layer "cluster" of type "Python" in function 'getLayerInstance'

Solution

Duplicate a .prototxt file to use it in OpenCV only. Replace the following layer:

layer {
    type: 'Python'
    name: 'cluster'
    bottom: 'coverage'
    bottom: 'bboxes'
    top: 'bbox-list'
    python_param {
        module: 'caffe.layers.detectnet.clustering'
        layer: 'ClusterDetections'
        param_str : '1248, 352, 16, 0.6, 3, 0.02, 22, 1'
    }
    include: { phase: TEST }
}

to

layer {
  name: 'cluster'
  type: 'ClusterDetections'
  bottom: 'coverage'
  bottom: 'bboxes'
  top: 'bbox-list'
  python_param {
    param_str: '1248, 352, 16, 0.6, 3, 0.02, 22, 1'
  }
}

Copy some usage from https://github.com/NVIDIA/caffe/blob/... to your script:

gridbox_to_boxes, vote_boxes, cluster methods without any changes

setup -> __init__ (modified)

reshape -> getMemoryShapes (modified)

forward -> forward (modified)

class ClusterDetections(object):
    def __init__(self, params, blobs):
        self.is_groundtruth = False
        try:
            plist = params['param_str'].split(',')
            self.image_size_x = int(plist[0])
            self.image_size_y = int(plist[1])
            self.stride = int(plist[2])
            self.gridbox_cvg_threshold = float(plist[3])
            self.gridbox_rect_thresh = int(plist[4])
            self.gridbox_rect_eps = float(plist[5])
            self.min_height = int(plist[6])
            self.num_classes = int(plist[7]) if len(plist) > 7 else 1
        except ValueError:
            raise ValueError("Parameter string missing or data type is wrong!")

    def getMemoryShapes(self, bottom):
        n_images = bottom[0][0]
        num_classes = bottom[0][1]
        if num_classes != self.num_classes:
            raise ValueError("Unexpected number of classes: %d != %d, bottom[0] shape=%s" % (num_classes, self.num_classes, repr(bottom[0].data.shape)))
        top = []
        for i in xrange(num_classes):
            top.append([1, n_images, MAX_BOXES, 5])  # Make it 4-dimensional
        return top

    def forward(self, bottom):
        top = []
        for i in xrange(self.num_classes):
            data0 = bottom[0][:,i:i+1,:,:]
            bbox = cluster(self, data0, bottom[1])
            top.append(np.expand_dims(bbox.astype(np.float32), 0))
        return top

cv.dnn_registerLayer('ClusterDetections', ClusterDetections)

TODO: I've tested that both OpenCV and Caffe returns similar values from coverage/sig and bbox/regressor layers using randomly initialized weights. I can test that detection post-processing also works good if you can share reference to trained .caffemodel because my returns just zeros.

P.S. Thanks to unsupported layer is the last you can just obtain both outputs and post-process them without a custom layer insertion.

edit flag offensive delete link more

Comments

Hi, I updated my OpenCV but I still get the error message saying:

OpenCV(3.4.1-dev) Error: Unspecified error (Can't create layer "cluster" of type "ClusterDetections") in getLayerInstance, file /opencv-master/modules/dnn/src/dnn.cpp, line 388 terminate called after throwing an instance of 'cv::Exception'

The problem is, I don't have a script to extend. The network I want to use was trained using Digits. I only have the .caffemodel, .binaryproto and the .prototxt files. But no script. Is there a way to get it from caffe based on these files? Or is it possible to define these layers in my C++ framework?

Sorry for asking dummy questions, but I am pretty new in DNN and I need a bit help to get everything to work...

Mary-Ann gravatar imageMary-Ann ( 2018-05-07 03:24:42 -0600 )edit

@Maty-Ann, that's OK. I just didn't know that you use C++ but not python. An origin implementation of this layer has been written in Python and it's the simplest solution to create a custom layer in python.

dkurt gravatar imagedkurt ( 2018-05-07 05:51:50 -0600 )edit

Hi, does it mean, that it is not possible currently to solve this problem inside my C++ framework? Unfortunately I am still not sure how to get this network work with OpenCV. Are you suggesting to modify the caffe script, retrain the network and use the new data in OpenCV?

Mary-Ann gravatar imageMary-Ann ( 2018-05-09 09:45:47 -0600 )edit

@Mary-Ann, you need to implement gridbox_to_boxes, vote_boxes, cluster methods in C++.

dkurt gravatar imagedkurt ( 2018-05-09 10:07:31 -0600 )edit

Hi, I switched to Python to test the application, but I still get some error. I was following this tutorial: https://www.pyimagesearch.com/2017/08... and I get the same output with the provided files. I extended the original code, just like you said, and register the new layer before I read the model and prototxt. But unfortunately I get the following error message: python/pyopencv_dnn.hpp:163: error: (-213:The function/feature is not implemented) Failed to call "forward" method in function 'forward' I am using OpenCV from trunk and updated and built everything today, but the error does not go away. Do you have any idea, how I could solve this? I am using Python 2.7 should I use 3.5 maybe?

Mary-Ann gravatar imageMary-Ann ( 2018-06-26 04:24:17 -0600 )edit

@dkurt your answer is correct and the application is finally running! I only needed to import math too. Thank you for your answer!

Mary-Ann gravatar imageMary-Ann ( 2018-06-27 08:46:03 -0600 )edit

hey guys i'm having troubles applying this solution can you please check this github issue that i posted, i would really appreciate it if you guide me through it link : https://github.com/NVIDIA/DIGITS/issu...

NidhamTekaya gravatar imageNidhamTekaya ( 2020-05-12 19:45:28 -0600 )edit

@dkurt did you solve it ? my returns are zeros too

NidhamTekaya gravatar imageNidhamTekaya ( 2020-05-13 08:16:42 -0600 )edit
0

answered 2018-07-04 07:04:36 -0600

You just remove or comment layer 'Python'

layer {
type: 'Python'
name: 'cluster'
bottom: 'coverage'
bottom: 'bboxes'
top: 'bbox-list'
python_param {
    module: 'caffe.layers.detectnet.clustering'
    layer: 'ClusterDetections'
    param_str : '1248, 352, 16, 0.6, 3, 0.02, 22, 1'
}
include: { phase: TEST }
}

to

# layer {

# type: 'Python'

# name: 'cluster'

# bottom: 'coverage'

# bottom: 'bboxes'

# top: 'bbox-list' # python_param {

# module: 'caffe.layers.detectnet.clustering'

# layer: 'ClusterDetections'

# param_str : '1248, 352, 16, 0.6, 3, 0.02, 22, 1'

# }

# include: { phase: TEST }

#}

I did that with python, but I dont know how to do with opencv c++, Can you share your experience with me?

edit flag offensive delete link more

Question Tools

1 follower

Stats

Asked: 2018-04-20 08:44:10 -0600

Seen: 1,062 times

Last updated: May 03 '18