DNN from Caffe deconvolution layer assert fails

asked 2017-09-26 17:49:33 -0500

Loca123 gravatar image

updated 2017-09-27 08:30:31 -0500

Hi,

I have a model which works fine in Caffe, but in OpenCV 3.3.0 it triggers an assert.

It fails because there is a convolution with 128 outputs followed by a ReLU and then a deconvolution with 64 outputs. OpenCV seems to have an assert stating that the number of outputs must match, however in Caffe this appears to be working fine.

So I was wondering if anyone else has come up against this issue and how they had got around it. Also, whether anyone knows why this isn't allowed in OpenCV?

Thanks for and insight!

Carl

Edit:

input: "images"
 input_shape {
   dim: 1
   dim: 3
   dim: 154
   dim: 100
 }
layer {
  name: "conv_d0a-b"
  type: "Convolution"
  bottom: "images"
  top: "d0b"
  convolution_param {
    num_output: 128
    kernel_size: 5
  }
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
}
layer {
  name: "relu_d0b"
  type: "ReLU"
  bottom: "d0b"
  top: "d0b"
}
layer {
  name: "upconv_u1d_u0a"
  type: "Deconvolution"
  bottom: "d0b"
  top: "d0b"
  convolution_param {
    num_output: 64
    kernel_size: 2
    stride: 2
  }
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
}
edit retag flag offensive close merge delete

Comments

Carl, the minimal .prototxt with mentioned sequence of layers would be better than any words ;)

dkurt gravatar imagedkurt ( 2017-09-27 00:46:43 -0500 )edit

Hi dkurt, thanks for the quick reply. I've added a simple network to the original post which reproduces the issue that I'm having. It triggers the assert:

 CV_Assert(!hasBias() || blobs[1].total() == (size_t)blobs[0].size[0]); in DeConvolutionLayerImpl::getMemoryShapes,

blobs[1].total() is 64 and blobs[0].size[0] is 128 at that point.

Loca123 gravatar imageLoca123 ( 2017-09-27 08:34:02 -0500 )edit

@Loca123, I can reproduce it. Thank you! Recently I've tried to run UNet model and there was the same problem. For now deconvolution kernel has shape out_channels X in_channels X height X widht. Expected that data is placed correspondingly. In this sample it turns out that deconvolution kernel has shape 128x64x2x2. I can't say right now if only shape values are swapped or kernel data is swapped too. I think we need some time to check that Caffe framework doesn't changed Deconvolution layer's kernel data layout sometimes and problem is just somewhere in DNN importer.

dkurt gravatar imagedkurt ( 2017-09-27 09:33:01 -0500 )edit

@Loca123, May I ask you to test a PR with fix? https://github.com/opencv/opencv/pull...

dkurt gravatar imagedkurt ( 2017-09-27 11:24:53 -0500 )edit

I've just tried it out but it's still hitting an assert. This time ln 810 in convolution_layer.cpp, CV_Assert(blobs[0].size[0] == inpCn);. This seems to be due to int outCn = blobs[0].size[0]; on line 176 being 64 instead of 128. I changed this to int outCn = numOutput; and it passed the assert but I had issues further down because the weights matrix size was incorrect. I'll try and have another look later and see if I can see where else the changes are needed.

p.s. thanks for looking into this so quickly

Loca123 gravatar imageLoca123 ( 2017-09-27 21:54:01 -0500 )edit

@Loca123, please try to reproduce these steps: (1) generate weights for corresponding network using Caffe, look at https://github.com/opencv/opencv/issu... for help. (2) Load model using DNN state @ PR https://github.com/opencv/opencv/pull.... Is there a problem?

dkurt gravatar imagedkurt ( 2017-09-28 01:46:53 -0500 )edit

Hi dkurt, silly mistake on my part last night, I was loading in my original model with the cut down network that I posted. I followed your steps and that worked fine, also my original model works fine too. Thanks so much for the help

Loca123 gravatar imageLoca123 ( 2017-09-28 04:36:20 -0500 )edit

@Loca123, thank you for the contribution!

dkurt gravatar imagedkurt ( 2017-09-28 05:24:17 -0500 )edit