DNN from Caffe deconvolution layer assert fails
Hi,
I have a model which works fine in Caffe, but in OpenCV 3.3.0 it triggers an assert.
It fails because there is a convolution with 128 outputs followed by a ReLU and then a deconvolution with 64 outputs. OpenCV seems to have an assert stating that the number of outputs must match, however in Caffe this appears to be working fine.
So I was wondering if anyone else has come up against this issue and how they had got around it. Also, whether anyone knows why this isn't allowed in OpenCV?
Thanks for and insight!
Carl
Edit:
input: "images"
input_shape {
dim: 1
dim: 3
dim: 154
dim: 100
}
layer {
name: "conv_d0a-b"
type: "Convolution"
bottom: "images"
top: "d0b"
convolution_param {
num_output: 128
kernel_size: 5
}
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
}
layer {
name: "relu_d0b"
type: "ReLU"
bottom: "d0b"
top: "d0b"
}
layer {
name: "upconv_u1d_u0a"
type: "Deconvolution"
bottom: "d0b"
top: "d0b"
convolution_param {
num_output: 64
kernel_size: 2
stride: 2
}
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
}
Carl, the minimal
.prototxtwith mentioned sequence of layers would be better than any words ;)Hi dkurt, thanks for the quick reply. I've added a simple network to the original post which reproduces the issue that I'm having. It triggers the assert:
blobs[1].total() is 64 and blobs[0].size[0] is 128 at that point.
@Loca123, I can reproduce it. Thank you! Recently I've tried to run UNet model and there was the same problem. For now deconvolution kernel has shape out_channels X in_channels X height X widht. Expected that data is placed correspondingly. In this sample it turns out that deconvolution kernel has shape 128x64x2x2. I can't say right now if only shape values are swapped or kernel data is swapped too. I think we need some time to check that Caffe framework doesn't changed Deconvolution layer's kernel data layout sometimes and problem is just somewhere in DNN importer.
@Loca123, May I ask you to test a PR with fix? https://github.com/opencv/opencv/pull...
I've just tried it out but it's still hitting an assert. This time ln 810 in convolution_layer.cpp, CV_Assert(blobs[0].size[0] == inpCn);. This seems to be due to int outCn = blobs[0].size[0]; on line 176 being 64 instead of 128. I changed this to int outCn = numOutput; and it passed the assert but I had issues further down because the weights matrix size was incorrect. I'll try and have another look later and see if I can see where else the changes are needed.
p.s. thanks for looking into this so quickly
@Loca123, please try to reproduce these steps: (1) generate weights for corresponding network using Caffe, look at https://github.com/opencv/opencv/issu... for help. (2) Load model using DNN state @ PR https://github.com/opencv/opencv/pull.... Is there a problem?
Hi dkurt, silly mistake on my part last night, I was loading in my original model with the cut down network that I posted. I followed your steps and that worked fine, also my original model works fine too. Thanks so much for the help
@Loca123, thank you for the contribution!