Wrong output for dnn using the master branch [closed]

asked 2017-11-17 10:53:51 -0600

Maups gravatar image

Hello everyone, thanks for all the help you gave me so far. If this problem is solved, this is probably my last question for a few months :)

I recently posted a problem that I had while running a tensorflow CNN using OpenCV, and I managed to make it run using the master branch thanks to dkurt. Now I need to make it work :P

The problem is that the master branch is giving numerical results very different from the version 3.3.0, which is the one giving results similar to tensorflow.

This is the code that I'm using: https://pastebin.com/xbcjru1m

This is an all-in-one implementation of our work that I intend to release soon (still not finished): https://arxiv.org/pdf/1710.07662.pdf

I added the tensorflow models and the image I use as input for the numerical examples below here: https://drive.google.com/open?id=1Vj2...

This is the output I get for version 3.3.0:

10.6368 -9.62331
57.1009,15.0345 49.1725,9.35289 41.2432,5.27753 33.1887,5.14996 26.9975,9.64316 24.1903,16.5573 23.3795,24.8945 24.703,32.8969 27.5521,40.898 30.6558,48.6363 33.6907,55.7202 36.9645,62.886 40.9232,69.6389 45.1071,76.0207 49.5653,82.1074 54.7589,87.2224 60.7443,91.2243 66.9655,92.4323 72.133,90.0989 75.1398,85.7536 60.6781,39.0338 56.7915,30.7083 52.2409,23.2375 46.7008,17.6615 40.4759,14.3339 33.8519,13.7421 29.0189,17.1593 28.1586,23.7111 28.9887,30.8795 31.2067,37.9041 33.5278,45.1232 36.3979,51.7835 39.2748,58.379 43.4146,64.2663 48.7502,69.3365 63.5614,42.5317 61.7706,48.4019 62.3384,56.5137 66.4177,61.6003 67.7452,66.2748 63.3672,64.6332 58.8021,60.4604 53.0287,59.6051 48.3177,56.8109 46.271,51.6537 46.3323,46.0759 47.5009,41.3566 49.7026,37.6776 52.3834,34.809 55.5715,32.9558 37.0859,45.7425 36.3689,38.8235 36.3324,31.8219 36.5529,24.9157 37.313,18.4695 
OpenCV Error: Assertion failed (srcMat.dims == 2 && srcMat.cols == weights.cols && dstMat.rows == srcMat.rows && dstMat.cols == weights.rows && srcMat.type() == weights.type() && weights.type() == dstMat.type() && srcMat.type() == CV_32F && (biasMat.empty() || (biasMat.type() == srcMat.type() && biasMat.isContinuous() && (int)biasMat.total() == dstMat.cols))) in run, file /home/mauricio/Libraries/OpenCV3.3/opencv-3.3.0/modules/dnn/src/layers/fully_connected_layer.cpp, line 132
terminate called after throwing an instance of 'cv::Exception'
  what():  /home/mauricio/Libraries/OpenCV3.3/opencv-3.3.0/modules/dnn/src/layers/fully_connected_layer.cpp:132: error: (-215) srcMat.dims == 2 && srcMat.cols == weights.cols && dstMat.rows == srcMat.rows && ...
(more)
edit retag flag offensive reopen merge delete

Closed for the following reason the question is answered, right answer was accepted by Maups
close date 2017-11-20 15:14:39.359987

Comments

1

@Maups, thank you for the interest in OpenCV! I think we have the same problem as described in referenced question: h_pool4_drop has NHWC data layout in TensorFlow but NCHW in OpenCV. After flattening we have two different vectors are multiplied to similar fully-connected layer's weights. As I promised, we'll make it smoother in future release but now I have to ask you to leave a small hint that helps OpenCV workaround it. Could you add an extra nodes before two reshapes that make nothing in TensorFlow but permute data in OpenCV: h_pool4_drop = tf.transpose(h_pool4_drop, [0, 1, 2, 3]). OpenCV maps it to [0, 2, 3, 1] (NCHW->NHWC) internally. Please tell us how it works.

dkurt gravatar imagedkurt ( 2017-11-17 12:02:57 -0600 )edit

I did it and got the following error message:

OpenCV Error: Assertion failed (ngroups > 0 && inpCn % ngroups == 0 && outCn % ngroups == 0) in getMemoryShapes, file /home/mauricio/Libraries/OpenCVmaster/opencv/modules/dnn/src/layers/convolution_layer.cpp, line 212
terminate called after throwing an instance of 'cv::Exception'
  what():  /home/mauricio/Libraries/OpenCVmaster/opencv/modules/dnn/src/layers/convolution_layer.cpp:212: error: (-215) ngroups > 0 && inpCn % ngroups == 0 && outCn % ngroups == 0 in function getMemoryShapes

The new classifier with this extra layer can be downloaded here: https://drive.google.com/open?id=1jYd...

Was it implemented differently in OpenCV 3.3.0? Do you know why do I get the correct result using 3.3.0?

Maups gravatar imageMaups ( 2017-11-17 14:19:24 -0600 )edit
1

@Maups, I think you placed it in a wrong place. I'm sorry for it. Please try again with (1) h_pool4_drop = tf.transpose(h_pool4_drop, [0, 1, 2, 3]) and flat_h_pool4_drop = tf.reshape(h_pool4_drop, [-1, np.prod(h_pool4_drop.shape[1:]).value]) (2) h_lconv3 = tf.transpose(h_lconv3, [0, 1, 2, 3])flat_h_lconv3 = tf.reshape(h_lconv3, [-1, np.prod(h_lconv3.shape[1:]).value]).

dkurt gravatar imagedkurt ( 2017-11-20 08:28:58 -0600 )edit

@dkurt Yes, my mistake. I put the line right after the max pooling instead of right before the reshape. It is working now, this is the link for the new description model, in case you need it: https://drive.google.com/open?id=1aMk... I still have to recreate the other models, but seems like I will be able to do it now. Thank you very much!

Maups gravatar imageMaups ( 2017-11-20 13:30:56 -0600 )edit
add a comment