Tensorflow/Keras model inference using OpenCV difficulties

asked 2020-04-24 19:47:58 -0500

Hi all:

I have made a neural network classification model using Keras (Tensorflow) backend.

The architecture is as below:

from keras.applications.mobilenet_v2 import MobileNetV2 
from keras.layers import MaxPooling2D, Dropout, Dense, Reshape, Permute
from keras.models import Sequential

base_model = MobileNetV2(include_top=False, weights='imagenet', input_shape = (224, 224, 3))  

model = Sequential () 
model.add (base_model)
model.add (MaxPooling2D())  
n, h, w, c = model.output_shape
n = c*h*w
model.add (Permute([3, 1, 2]))  # Indicate NHWC data layout
model.add (Reshape((n,)))
model.add (Dropout(0.92))   
model.add (Dense(2, activation='sigmoid')) 

model.summary ()

Note: since I read OpenCV can't handle the "flatten" layer (and experimented), I replaced it with "permute" and "reshape" layer following instructions here. This works fine, I got 95%+ val accuracy.

I then converted the HDF5 format of the Keras model to .pb graph using this script that I found, this seems to work:

from tensorflow.python.tools import freeze_graph
from tensorflow.python.tools import optimize_for_inference_lib

MODEL_PATH = 'keras_test'
MODEL_NAME = 'test'
input_node_name = 'mobilenetv2_1.00_224_input'
output_node_name = 'dense_18/Sigmoid'

tf.train.write_graph(sess.graph_def, MODEL_PATH, f'{MODEL_NAME}_graph.pb', as_text=False)
tf.train.write_graph(sess.graph_def, MODEL_PATH, f'{MODEL_NAME}_graph.pbtxt')
tf.train.Saver().save(sess, f'{MODEL_PATH}/{MODEL_NAME}.chkp')

freeze_graph.freeze_graph(f'{MODEL_PATH}/{MODEL_NAME}_graph.pbtxt',
                          None, False,
                          f'{MODEL_PATH}/{MODEL_NAME}.chkp',
                          output_node_name,
                          "save/restore_all",
                          "save/Const:0",
                          f'{MODEL_PATH}/frozen_{MODEL_NAME}.pb', 
                          True, "")

The inference code is as below:

import cv2 as cv
cvNet = cv.dnn.readNetFromTensorflow('test_graph.pb')
img = cv.imread('some_img.jpg')
cvNet.setInput(cv.dnn.blobFromImage(img, size=(300, 300), swapRB=True, crop=False))
dnn_out = cvNet.forward()

But I keep getting this message when doing inference:

error: OpenCV(4.1.2) /io/opencv/modules/dnn/src/layers/fully_connected_layer.cpp:154: error: (-215:Assertion failed) srcMat.dims == 2 && srcMat.cols == weights.cols && dstMat.rows == srcMat.rows && dstMat.cols == weights.rows && srcMat.type() == weights.type() && weights.type() == dstMat.type() && srcMat.type() == CV_32F && (biasMat.empty() || (biasMat.type() == srcMat.type() && biasMat.isContinuous() && (int)biasMat.total() == dstMat.cols)) in function 'run'

Any advice would be appreciated, thanks in advance.

edit retag flag offensive close merge delete

Comments

But, did readNetFromTensorflow() function read correctly the model or its an issue of the forward pass?

FilipVW gravatar imageFilipVW ( 2020-04-25 08:13:36 -0500 )edit