Keras -> TensorFlow -> OpenCV/dnn
I try to use a Tensorflow model (created with Keras) in OpenCV/dnn (3.3.0). But I get the following error message:
OpenCV Error: Unspecified error (Unknown layer type Shape in op flatten_1/Shape)
This is the python code to create and export the model.
import os
import sys
import shutil
import subprocess
import numpy as np
import tensorflow as tf
from keras.models import Model
from keras import backend as K
from keras.layers import Dense, Input, Conv2D, Flatten, MaxPooling2D, BatchNormalization
sess = tf.Session()
K.set_session(sess)
# create model
inputs = Input(shape=(64, 64, 3), name='input_layer')
x = Conv2D(8, (3, 3), padding='same', activation='relu')(inputs)
x = MaxPooling2D((2,2))(x)
x = Conv2D(16, (3, 3), padding='same', activation='relu')(x)
x = MaxPooling2D((2,2))(x)
x = Flatten()(x)
x = Dense(64, activation='sigmoid')(x)
x = BatchNormalization()(x)
x = Dense(32, activation='sigmoid')(x)
predictions = Dense(10, activation='softmax', name='output_layer')(x)
model = Model(inputs=inputs, outputs=predictions)
model.compile(optimizer='rmsprop',
loss='categorical_crossentropy',
metrics=['accuracy'])
# train model
data = np.random.random((100, 64, 64, 3))
labels = np.random.randint(2, size=(100, 10))
model.fit(data, labels, epochs=10, batch_size=32)
# create temp directory for export
temp_export_dir = './temp_export/'
if os.path.exists(temp_export_dir):
shutil.rmtree(temp_export_dir)
os.makedirs(temp_export_dir)
# export model to temp directory
with sess.graph.as_default():
saver = tf.train.Saver()
saver.save(sess, temp_export_dir + 'model.ckpt')
tf.train.write_graph(sess.graph.as_graph_def(), temp_export_dir, "graph.pbtxt")
# freeze graph and optimize for inference
_ = subprocess.call("python freeze_graph.py --input_graph=" + temp_export_dir + "graph.pbtxt --input_checkpoint=" + temp_export_dir + "model.ckpt --output_graph=" + temp_export_dir + "frozen_graph.pb --output_node_names=output_layer/Softmax")
_ = subprocess.call("python optimize_for_inference.py --input=" + temp_export_dir + "frozen_graph.pb --output=model.pb --frozen_graph=True --input_names=input_layer --output_names=output_layer/Softmax")
# remove temp directory
#shutil.rmtree(temp_export_dir)
The two external scripts freeze_graph.py
and optimize_for_inference.py
come from tensorflow/python/tools
And this is the C++ code to load the model:
#include <opencv2/dnn.hpp>
int main()
{
cv::dnn::Net net = cv::dnn::readNetFromTensorflow( "model.pb" );
}
I uploaded the file model.pb
here: http://daiw.de/share/opencv_question_...
Am I doing something wrong or can OpenCV/dnn simply not import such a model?
Hi, @Dobiasd! Please, see https://github.com/opencv/opencv_cont....
Hi @dkurtaev. Thanks for the quick respone.
wilcoschoneveld uses Keras via
tensorflow.contrib.keras
. But my Keras model is backend agnostic. So I guesstf.reshape
would not be an optinal solution/workaround in my case.However, as a test I removed the
Flatten
part completely: https://ideone.com/ZhXwhT This results inOpenCV Error: Unspecified error (Unknown layer type Mean in op batch_normalization_1/moments/Mean)
Using
selu
instead ofrelu
as the activation in a conv layer results inOpenCV Error: Unspecified error (Unknown layer type Greater in op conv2d_1/Greater)
Do you know if it is planned to support all these things in the future? And if so is there already a roadmap?
We support batch normalization which was serialized in fused mode. You may find test case here: https://github.com/opencv/opencv_extr.... Instead
is_training=False
you may replaceFalse
by variable and set toTrue
during training andFalse
before serialization.OK, thanks. However it looks like the Keras interface does not provide these fine-grained options. Is it planned to support Keras models natively without going through the indirection of another model format like TensorFlow's?
Hi @Dobiasd, I'm running your script above but It looks like it failed at freeze_graph.py. Could you let me know which version of tensorflow you were using?