Ask Your Question
1

Opencv dnn import error for keras pretrained Vgg16 model

asked 2018-01-30 02:00:16 -0500

MennoK gravatar image

updated 2018-01-30 07:03:34 -0500

dkurt gravatar image

Hi,

I'm trying to export the keras pretrained VGG16 model to a tensorflow model and then I want to import the tensorflow model into opencv. To do this, I got the following python code:

from keras import applications
from keras import backend as K
import cv2 as cv
import tensorflow as tf
from tensorflow.python.framework import graph_util
from tensorflow.python.framework import graph_io

model = applications.VGG16(input_shape=(224, 224, 3), weights='imagenet', include_top=True)
K.set_learning_phase(0)

pred_node_names = [None]
pred = [None]
for i in range(1):
    pred_node_names[i] = "output_node"+str(i)
    pred[i] = tf.identity(model.outputs[i], name=pred_node_names[i])

 sess = K.get_session()
 constant_graph = graph_util.convert_variables_to_constants(sess, sess.graph.as_graph_def(), pred_node_names)
 graph_io.write_graph(constant_graph, ".", "model.pb", as_text=False)

net = cv.dnn.readNetFromTensorflow('model.pb')

Executing this code, gives following error (similar in C++):

cv2.error: C:\projects\opencv-python\opencv\modules\dnn\src\tensorflow\tf_importer.cpp:1487: error: (-2) Unknown layer type Shape in op flatten/Shape in function cv::dnn::experimental_dnn_v3::`anonymous-namespace'::TFImporter::populateNet

When I'm not including the top, I don't receive this error message.

Do you have any suggestions?

edit retag flag offensive close merge delete

Comments

@MennoK, please attach a mentioned .pb file.

dkurt gravatar imagedkurt ( 2018-01-30 02:45:06 -0500 )edit
1

The .pb file is large (+500mb), so here is a google drive link: https://drive.google.com/file/d/11O-J...

MennoK gravatar imageMennoK ( 2018-01-30 05:45:14 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
3

answered 2018-01-30 07:02:47 -0500

dkurt gravatar image

updated 2018-01-30 07:05:06 -0500

@MennoK, You may see that TensorFlow makes a flattening through Reshape layer using dynamically calculated shape:

import tensorflow as tf

# Read the graph.
with tf.gfile.FastGFile('model.pb') as f:
    graph_def = tf.GraphDef()
    graph_def.ParseFromString(f.read())

with tf.Session() as sess:
    # Restore session
    sess.graph.as_default()
    tf.import_graph_def(graph_def, name='')
    tf.summary.FileWriter('logs', graph_def)

image description

OpenCV can't do it. There are two possible ways: replace a flatten node by reshape during graph creation or help OpenCV to import it using text graph.

Optimize

Run optimize_for_inference.py tool to remove some extra nodes from the graph:

python ~/tensorflow/tensorflow/python/tools/optimize_for_inference.py \
  --input model.pb \
  --output opt_model.pb \
  --input_names input_1 \
  --output_names predictions/Softmax

Text graph

Run the following script to make a text graph representation:

import tensorflow as tf

# Read the graph.
with tf.gfile.FastGFile('opt_model.pb') as f:
    graph_def = tf.GraphDef()
    graph_def.ParseFromString(f.read())

# Remove Const nodes.
for i in reversed(range(len(graph_def.node))):
    if graph_def.node[i].op == 'Const':
        del graph_def.node[i]
    for attr in ['T', 'data_format', 'Tshape', 'N', 'Tidx', 'Tdim',
                 'use_cudnn_on_gpu', 'Index', 'Tperm', 'is_training',
                 'Tpaddings']:
        if attr in graph_def.node[i].attr:
            del graph_def.node[i].attr[attr]

# Save as text.
tf.train.write_graph(graph_def, "", "model.pbtxt", as_text=True)

Flatten

Open model.pbtxt and remove nodes with names flatten/Shape, flatten/strided_slice, flatten/Prod, flatten/stack. Replace a node

node {
  name: "flatten/Reshape"
  op: "Reshape"
  input: "block5_pool/MaxPool"
  input: "flatten/stack"
}

on

node {
  name: "flatten/Reshape"
  op: "Flatten"
  input: "block5_pool/MaxPool"
}

Launch

Now you can remove opt_model.pb and use both model.pb and model.pbtxt in OpenCV:

import cv2 as cv
import numpy as np
net = cv.dnn.readNetFromTensorflow('model.pb', 'model.pbtxt')

inp = np.random.standard_normal([1, 3, 224, 224]).astype(np.float32)
net.setInput(inp)
out = net.forward()
edit flag offensive delete link more

Comments

Thanks a lot for the answer. However I'm not able to make the text graph as you described. I get following error on the line

graph_def.ParseFromString(f.read())

Error message: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 38: invalid continuation type.

Which version of Python and Tensorflow are you running? I'm running 3.5.4 and 1.4.0 respectively.

MennoK gravatar imageMennoK ( 2018-01-30 07:20:45 -0500 )edit
1

I fixed the error by changing line:

with tf.gfile.FastGFile('opt_model.pb') as f:

to

with tf.gfile.FastGFile('opt_model.pb', 'rb') as f:
MennoK gravatar imageMennoK ( 2018-01-30 07:27:18 -0500 )edit

Thanks a lot again. Now, I'm also able to import the model in open cv C++

MennoK gravatar imageMennoK ( 2018-01-30 07:50:17 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-01-30 02:00:16 -0500

Seen: 1,573 times

Last updated: Jan 30 '18