Export tensorflow graph with batchnorm to opencv dnn

asked 2017-09-29 20:25:32 -0500

tham gravatar image

updated 2017-10-02 04:15:59 -0500

First, describe a net with batchnorm

    net = tf.layers.conv2d(inputs = features, filters = 64, kernel_size = [3, 3], strides = (2, 2), padding = 'same')
    training = tf.Variable(False, name = 'training')
    net = tf.contrib.layers.batch_norm(net, is_training = training)
    net = tf.nn.relu(net)
    net = tf.reshape(net, [-1, 64 * 7 * 7]) #
    net = tf.layers.dense(inputs = net, units = class_num, kernel_initializer = tf.contrib.layers.xavier_initializer(), name = 'regression_output')


#......
#after training, save the graph and weights

sess.run(loss, feed_dict={features : train_imgs, x : real_delta, training : False})
saver = tf.train.Saver()
saver.save(sess, 'reshape_final.ckpt')
tf.train.write_graph(sess.graph.as_graph_def(), "", 'graph_final.pb')

After that, I freeze the graph->optimize>transform

python3 ~/.keras2/lib/python3.5/site-packages/tensorflow/python/tools/freeze_graph.py --input_graph=graph_final.pb --input_checkpoint=reshape_final.ckpt --output_graph=frozen_graph.pb --output_node_names=regression_output/BiasAdd

python3 ~/.keras2/lib/python3.5/site-packages/tensorflow/python/tools/optimize_for_inference.py --input frozen_graph.pb --output opt_graph.pb --frozen_graph True --input_names input --output_names regression_output/BiasAdd

~/Qt/3rdLibs/tensorflow/bazel-bin/tensorflow/tools/graph_transforms/transform_graph --in_graph=opt_graph.pb --out_graph=fused_graph.pb --inputs=input --outputs=regression_output/BiasAdd --transforms="fold_constants fold_batch_norms fold_old_batch_norms sort_by_execution_order"

Load the model by opencv dnn,

std::string const model("/home/ramsus/Qt/blogCodes2/deep_homography/cnn/tensorflow/fused_graph.pb");

dnn::Net net = dnn::readNetFromTensorflow(model);
if(net.empty()){
    std::cerr<<"Can't load network by using the mode file:"<<std::endl;
    std::cerr<<model<<std::endl;
    throw std::runtime_error("net is empty");
}

it throw error messages:

BatchNorm/moments/mean:Mean(conv2d/convolution)(BatchNorm/moments/mean/reduction_indices) keep_dims:[ ] Tidx:[ ] T:0 OpenCV Error: Unspecified error (Unknown layer type Mean in op BatchNorm/moments/mean) in populateNet, file /home/ramsus/Qt/3rdLibs/opencv/modules/dnn/src/tensorflow/tf_importer.cpp, line 1077 /home/ramsus/Qt/3rdLibs/opencv/modules/dnn/src/tensorflow/tf_importer.cpp:1077: error: (-2) Unknown layer type Mean in op BatchNorm/moments/mean in function populateNet

edit retag flag offensive close merge delete

Comments

Do anyone know this is my fault or the limitation of opencv dev? Thanks

tham gravatar imagetham ( 2017-10-03 15:47:05 -0500 )edit

@tham, there are several ways: (1) save a checkpoint as usual but graph definition with is_training=False, (2) save graph definition in text, remove all weights and Const ops, replace subgraph that manages different batch normalization behavior to single FusedBatchNorm node, see https://github.com/opencv/opencv_extr... and https://github.com/opencv/opencv_extr... with PR https://github.com/opencv/opencv/pull....

dkurt gravatar imagedkurt ( 2017-10-04 23:19:35 -0500 )edit

@dkurt Thanks, I will give solution 1 a shot. Solution 2, I think I will try it after they are merged. Would torch model easier to parse compare with tensorflow? Thnaks for your helps, it must be very hard to parse a complex graph generated by tensorflow.

tham gravatar imagetham ( 2017-10-04 23:45:24 -0500 )edit