Ask Your Question
0

(DNN) different results between version 3.3.0 and 3.3.1

asked 2017-12-15 06:14:15 -0500

Rodrigo2019 gravatar image

System information (version)

OpenCV => 3.3.0/3.3.1

Operating System / Platform => Windows 10 64 Bit

Compiler => Visual Studio 2015

Detailed description

I have a network that works fine in Opencv 3.3.0, but after updating my opencv to the version 3.3.1 I'm getting wrong results with the same code. What I already tried:

*Compile on Linux -> I got the same wrong results

*Compile on windows with Mingw -> I got the same wrong results

*Compile on windows with Visual Studio 14 x32 -> I got the same wrong results

*Compile the master brach of opencv on windows with Visual Studio 14 x32 -> I got the same wrong results

Complementar tests: I used the "tensorflow_inception_graph.pb" network, with this network I got the same results in version 3.3.0 and 3.3.1, I do not know if it is a correct predictions. Using the caffe model network from the opencv examples worked as well with correct prediction for both versions.

Maybe my problem is my network, but why my network works on opencv 3.3.0 and dont work on 3.3.1?

Steps to reproduce

NetworkInput: 1x1x28x92 (grayscale image)

Normalization: 0..1

The same code is used in opencv 3.3.0 and 3.3.1

my network you can find here

#include <opencv2/dnn.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/core/utils/trace.hpp>
//using namespace cvtest;
using namespace cv;
using namespace cv::dnn;


#include <fstream>
#include <iostream>
#include <cstdlib>
using namespace std;

static void getMaxClass(const Mat &probBlob, int *classId, double *classProb)
{
    Mat probMat = probBlob.reshape(1, 1); //reshape the blob to 1x1000 matrix
    Point classNumber;
    minMaxLoc(probMat, NULL, classProb, NULL, &classNumber);
    *classId = classNumber.x;
}
int main()
{

    //CV_TRACE_FUNCTION();
    String modelBin = "model_final.pb";
    String imageFile = "airplane.jpg";
    Net net = dnn::readNetFromTensorflow(modelBin);
    if (net.empty())
    {
        std::cerr << "Can't load network by using the following files: " << std::endl;
        std::cerr << "Tensorflow model: " << modelBin << std::endl;
        exit(-1);
    }

    Mat img = imread(imageFile,0);
    if (img.empty())
    {
        std::cerr << "Can't read image from the file: " << imageFile << std::endl;
        exit(-1);
    }
    Mat resized;
    resize(img, resized, Size(92, 28));
    float escala=1.0/255.0;
    Mat inputBlob = blobFromImage(resized,escala, Size(92, 28),Scalar(0,0,0),false);   //Convert Mat to batch of images
    std::cout << inputBlob.size << std::endl;
    Mat prob;

    cv::TickMeter t;
    for (int i = 0; i < 100; i++)
    {
        //CV_TRACE_REGION("forward");
        t.start();
        net.setInput(inputBlob, "conv2d_1_input");        //set the network input
        prob = net.forward("activation_4/Softmax");                          //compute output
        //std::cout << prob << std::endl;
        t.stop();
    }
    int classId;
    double classProb;
    getMaxClass(prob, &classId, &classProb);//find the best class

    std::cout << prob<< std::endl;
    std::cout << "Best class: #" << classId << std::endl;
    std::cout << "Probability: " << classProb * 100 << "%" << std::endl;
    std::cout << "Time: " << (double)t.getTimeMilli() / t.getCounter() << " ms (average ...
(more)
edit retag flag offensive close merge delete

Comments

Hi Rodrigo2019 How are you?

Can you tell me how you managed the network input like this in training model NetworkInput: 1x1x28x92 (grayscale image)

I am also facing the problem with NHWC data layout to NCHW.

How can I change my training model to get the right result from the opencv.

I also posted my question in detail

http://answers.opencv.org/question/18...

Kindly help me in this regard.

Thank you so much

mindHunter099 gravatar imagemindHunter099 ( 2017-12-19 02:59:03 -0500 )edit

I create my batchs doing something like this:

im=cv2.imread("example.jpg",0) #It will load in grayscale
im=im/255.0 #scale 0..1
im=cv2.resize(im,(92,28))
im=im[np.newaxis,:,:,np.newaxis] #create a bacth with dimensions 1x28x92x1

importing the model with opencv dnn module, your model will automaticly transformed to NCHW, and using the function cv2.blobFromImage will create a batch with dimensions NCHW

Rodrigo2019 gravatar imageRodrigo2019 ( 2017-12-20 12:45:43 -0500 )edit

2 answers

Sort by ยป oldest newest most voted
2

answered 2017-12-18 09:02:42 -0500

dkurt gravatar image

@Rodrigo2019, I'm sorry for the delay. The problem is that at OpenCV 3.3.0 reshape layer had a kind of condition to make a data permutation (like from NHWC data layout to NCHW). We've removed it. Shortly your model is a sequence of the following layers:

Placeholder conv2d_1_input
Const conv2d_1/kernel
Conv2D conv2d_1/convolution
Const conv2d_1/bias
BiasAdd conv2d_1/BiasAdd
Relu activation_1/Relu
MaxPool max_pooling2d_1/MaxPool
Const conv2d_2/kernel
Conv2D conv2d_2/convolution
Const conv2d_2/bias
BiasAdd conv2d_2/BiasAdd
Relu activation_2/Relu
MaxPool max_pooling2d_2/MaxPool
Const reshape_1/Reshape/shape
Reshape reshape_1/Reshape
Const dense_1/kernel
MatMul dense_1/MatMul
Const dense_1/bias
BiasAdd dense_1/BiasAdd
Relu activation_3/Relu
Const dense_2/kernel
MatMul dense_2/MatMul
Const dense_2/bias
BiasAdd dense_2/BiasAdd
Softmax activation_4/Softmax

And we have to add one permutation manually to indicate that fully-connected layer expects NHWC input (default TensorFlow data layout).

The easiest way is to use text graph representation. To do it just run the following script:

import tensorflow as tf

# Read the graph.
with tf.gfile.FastGFile('model_final.pb') as f:
    graph_def = tf.GraphDef()
    graph_def.ParseFromString(f.read())

# Remove Const nodes.
for i in reversed(range(len(graph_def.node))):
    if graph_def.node[i].op == 'Const':
        del graph_def.node[i]
    for attr in ['T', 'data_format', 'Tshape', 'N', 'Tidx', 'Tdim',
                 'use_cudnn_on_gpu', 'Index', 'Tperm', 'is_training',
                 'Tpaddings']:
        if attr in graph_def.node[i].attr:
            del graph_def.node[i].attr[attr]

# Save as text.
tf.train.write_graph(graph_def, "", "model_final.pbtxt", as_text=True)

And replace

node {
  name: "reshape_1/Reshape"
  op: "Reshape"
  input: "max_pooling2d_2/MaxPool"
  input: "reshape_1/Reshape/shape"
}

onto

node {
  name: "order"
  op: "Const"
  attr {
    key: "value"
    value {
      tensor {
        dtype: DT_INT32
        tensor_shape {
        }
        int_val: 0
        int_val: 1
        int_val: 2
        int_val: 3
      }
    }
  }
}
node {
  name: "max_pooling2d_2/MaxPool/transpose"
  op: "Transpose"
  input: "max_pooling2d_2/MaxPool"
  input: "order"
}
node {
  name: "reshape_1/Reshape"
  op: "Reshape"
  input: "max_pooling2d_2/MaxPool/transpose"
  input: "reshape_1/Reshape/shape"
}

Now the following script that matches TensorFlow's output with OpenCV's one produces 2.91038e-11 maximal absolute difference.

import numpy as np
import tensorflow as tf
import time

with tf.gfile.FastGFile('model_final.pb') as f:
    graph_def = tf.GraphDef()
    graph_def.ParseFromString(f.read())

with tf.Session() as sess:
    sess.graph.as_default()
    tf.import_graph_def(graph_def, name='')
    for node in graph_def.node:
        print node.op, node.name

    # Generate input
    np.random.seed(2701)
    inp = np.random.standard_normal([1, 28, 92, 1]).astype(np.float32)

    # Get output tensor
    outTensor = sess.graph.get_tensor_by_name('activation_4/Softmax:0')
    out = sess.run(outTensor, feed_dict={'conv2d_1_input:0': inp})

import cv2 as cv
cvNet = cv.dnn.readNetFromTensorflow('model_final.pb', 'model_final.pbtxt')
cvNet.setInput(inp.transpose(0, 3, 1, 2))
cvOut = cvNet.forward()
print np.max(np.abs(cvOut - out))
edit flag offensive delete link more
0

answered 2017-12-19 10:28:50 -0500

Rodrigo2019 gravatar image

Thank you so much Dkurt, I would never found this answer by myself. I will test as soon as possible, thank you

edit flag offensive delete link more

Comments

@Rodrigo2019, We have good news about that problem. PR https://github.com/opencv/opencv/pull... was proposed to let you don't create a text graph for this test case. May I ask you to test it? The only difference is that you no longer need model_final.pbtxt: just call cvNet = cv.dnn.readNetFromTensorflow('model_final.pb').

dkurt gravatar imagedkurt ( 2017-12-20 05:31:17 -0500 )edit

for sure, I will test it tomorrow morning and I post the results here

Rodrigo2019 gravatar imageRodrigo2019 ( 2017-12-20 12:46:50 -0500 )edit
1

@dkurt, I tested what did you said. First I downloaded the last version of opencv-master and replaced with your files Testing on windows with visual studio Release mode x86, the predictions now is correct Also my other networks that was causing crashs in opencv now is working too, these networks was using batchNorm and leakyRelu (keras 2.0.8)

Rodrigo2019 gravatar imageRodrigo2019 ( 2017-12-21 06:44:00 -0500 )edit

@dkurt, thanks a lot, your modifications will help me a lot

Rodrigo2019 gravatar imageRodrigo2019 ( 2017-12-21 06:45:41 -0500 )edit

@Rodrigo2019, that's perfect! Many thanks! It's very helpful for us to improve users' experience of using OpenCV.

dkurt gravatar imagedkurt ( 2017-12-21 06:51:32 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-12-15 06:14:15 -0500

Seen: 413 times

Last updated: Dec 19 '17