Keras -> TensorFlow -> OpenCV/dnn

asked 2017-08-23 10:40:33 -0600

Dobiasd gravatar image

I try to use a Tensorflow model (created with Keras) in OpenCV/dnn (3.3.0). But I get the following error message:

OpenCV Error: Unspecified error (Unknown layer type Shape in op flatten_1/Shape)

This is the python code to create and export the model.

import os
import sys
import shutil
import subprocess

import numpy as np

import tensorflow as tf

from keras.models import Model
from keras import backend as K

from keras.layers import Dense, Input, Conv2D, Flatten, MaxPooling2D, BatchNormalization


sess = tf.Session()
K.set_session(sess)

# create model
inputs = Input(shape=(64, 64, 3), name='input_layer')
x = Conv2D(8, (3, 3), padding='same', activation='relu')(inputs)
x = MaxPooling2D((2,2))(x)
x = Conv2D(16, (3, 3), padding='same', activation='relu')(x)
x = MaxPooling2D((2,2))(x)
x = Flatten()(x)
x = Dense(64, activation='sigmoid')(x)
x = BatchNormalization()(x)
x = Dense(32, activation='sigmoid')(x)
predictions = Dense(10, activation='softmax', name='output_layer')(x)
model = Model(inputs=inputs, outputs=predictions)
model.compile(optimizer='rmsprop',
              loss='categorical_crossentropy',
              metrics=['accuracy'])

# train model
data = np.random.random((100, 64, 64, 3))
labels = np.random.randint(2, size=(100, 10))
model.fit(data, labels, epochs=10, batch_size=32)

# create temp directory for export
temp_export_dir = './temp_export/'
if os.path.exists(temp_export_dir):
    shutil.rmtree(temp_export_dir)
os.makedirs(temp_export_dir)

# export model to temp directory
with sess.graph.as_default():
    saver = tf.train.Saver()
    saver.save(sess, temp_export_dir + 'model.ckpt')
    tf.train.write_graph(sess.graph.as_graph_def(), temp_export_dir, "graph.pbtxt")

# freeze graph and optimize for inference
_ = subprocess.call("python freeze_graph.py --input_graph=" + temp_export_dir + "graph.pbtxt --input_checkpoint=" + temp_export_dir + "model.ckpt --output_graph=" + temp_export_dir + "frozen_graph.pb --output_node_names=output_layer/Softmax")
_ = subprocess.call("python optimize_for_inference.py --input=" + temp_export_dir + "frozen_graph.pb --output=model.pb --frozen_graph=True --input_names=input_layer --output_names=output_layer/Softmax")

# remove temp directory
#shutil.rmtree(temp_export_dir)

The two external scripts freeze_graph.py and optimize_for_inference.py come from tensorflow/python/tools

And this is the C++ code to load the model:

#include <opencv2/dnn.hpp>
int main()
{
    cv::dnn::Net net = cv::dnn::readNetFromTensorflow( "model.pb" );
}

I uploaded the file model.pbhere: http://daiw.de/share/opencv_question_...

Am I doing something wrong or can OpenCV/dnn simply not import such a model?

edit retag flag offensive close merge delete

Comments

dkurtaev gravatar imagedkurtaev ( 2017-08-24 00:27:03 -0600 )edit

Hi @dkurtaev. Thanks for the quick respone.

wilcoschoneveld uses Keras via tensorflow.contrib.keras. But my Keras model is backend agnostic. So I guess tf.reshape would not be an optinal solution/workaround in my case.

However, as a test I removed the Flattenpart completely: https://ideone.com/ZhXwhT This results in OpenCV Error: Unspecified error (Unknown layer type Mean in op batch_normalization_1/moments/Mean)

Using selu instead of relu as the activation in a conv layer results in OpenCV Error: Unspecified error (Unknown layer type Greater in op conv2d_1/Greater)

Do you know if it is planned to support all these things in the future? And if so is there already a roadmap?

Dobiasd gravatar imageDobiasd ( 2017-08-24 08:01:47 -0600 )edit

We support batch normalization which was serialized in fused mode. You may find test case here: https://github.com/opencv/opencv_extr.... Instead is_training=False you may replace False by variable and set to True during training and False before serialization.

dkurtaev gravatar imagedkurtaev ( 2017-08-24 08:34:16 -0600 )edit

OK, thanks. However it looks like the Keras interface does not provide these fine-grained options. Is it planned to support Keras models natively without going through the indirection of another model format like TensorFlow's?

Dobiasd gravatar imageDobiasd ( 2017-08-24 09:53:06 -0600 )edit

Hi @Dobiasd, I'm running your script above but It looks like it failed at freeze_graph.py. Could you let me know which version of tensorflow you were using?

oishi89 gravatar imageoishi89 ( 2017-10-26 02:44:45 -0600 )edit