# Revision history [back]

### OpenCV predictions much worse compared to Tensorflow

I have a ResNet type of model, that I have simplified and that I would like to use with OpenCV.

Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
input_1 (InputLayer)            [(None, 50, 50, 1)]  0
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 50, 50, 1)    4           input_1[0][0]
__________________________________________________________________________________________________
conv2d (Conv2D)                 (None, 50, 50, 20)   200         batch_normalization[0][0]
__________________________________________________________________________________________________
re_lu (ReLU)                    (None, 50, 50, 20)   0           conv2d[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 50, 50, 20)   80          re_lu[0][0]
__________________________________________________________________________________________________

...

conv2d_13 (Conv2D)              (None, 13, 13, 80)   57680       batch_normalization_12[0][0]
__________________________________________________________________________________________________
conv2d_13[0][0]
__________________________________________________________________________________________________
re_lu_12 (ReLU)                 (None, 13, 13, 80)   0           add_5[0][0]
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, 13, 13, 80)   320         re_lu_12[0][0]
__________________________________________________________________________________________________
average_pooling2d (AveragePooli (None, 3, 3, 80)     0           batch_normalization_13[0][0]
__________________________________________________________________________________________________
flatten (Flatten)               (None, 720)          0           average_pooling2d[0][0]
__________________________________________________________________________________________________
dense (Dense)                   (None, 2)            1442        flatten[0][0]
==================================================================================================


I followed the tutorial to freeze the model and to optimize it using Tensorflow 1.5 tensorflow.python.tools.optimize_for_inference tool.

• https://medium.com/@sebastingarcaacosta/how-to-export-a-tensorflow-2-x-keras-model-to-a-frozen-and-optimized-graph-39740846d9eb

It works great and OpenCV brings 10 times better performance! Just amazing!

But comparing the prediction results between Tensorflow (not freezed) model with OpenCV (freezed and optimized) model I found, that a large percentage (5% or 10%) of predictions differs by a lot. Some predictions even flips from Yes to No. Like an example here

OpenCV: [[0.4270926  0.57290745]]
TF    : [[0.9662978  0.03370225]]


My questions:

• What is the reason for this inaccuracy?

• Can I get my accuracy back by losing some of the amazing performance?

• May it be the precision drop from float32 to float16 and if yes, how can I force OpenCV to use float32?

### OpenCV predictions much worse compared to Tensorflow

I have a ResNet type of model, that I have simplified and that I would like to use with OpenCV.

Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
input_1 (InputLayer)            [(None, 50, 50, 1)]  0
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 50, 50, 1)    4           input_1[0][0]
__________________________________________________________________________________________________
conv2d (Conv2D)                 (None, 50, 50, 20)   200         batch_normalization[0][0]
__________________________________________________________________________________________________
re_lu (ReLU)                    (None, 50, 50, 20)   0           conv2d[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 50, 50, 20)   80          re_lu[0][0]
__________________________________________________________________________________________________

...

conv2d_13 (Conv2D)              (None, 13, 13, 80)   57680       batch_normalization_12[0][0]
__________________________________________________________________________________________________
conv2d_13[0][0]
__________________________________________________________________________________________________
re_lu_12 (ReLU)                 (None, 13, 13, 80)   0           add_5[0][0]
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, 13, 13, 80)   320         re_lu_12[0][0]
__________________________________________________________________________________________________
average_pooling2d (AveragePooli (None, 3, 3, 80)     0           batch_normalization_13[0][0]
__________________________________________________________________________________________________
flatten (Flatten)               (None, 720)          0           average_pooling2d[0][0]
__________________________________________________________________________________________________
dense (Dense)                   (None, 2)            1442        flatten[0][0]
==================================================================================================


I followed the tutorial to freeze the model and to optimize it using Tensorflow 1.5 tensorflow.python.tools.optimize_for_inference tool.

• https://medium.com/@sebastingarcaacosta/how-to-export-a-tensorflow-2-x-keras-model-to-a-frozen-and-optimized-graph-39740846d9eb

My code:

# Load Tensorflow model

# Normalize image
IMG_SIZE = 50
def normalize(img: np.ndarray) -> np.ndarray:
blob = cv2.resize(img, (IMG_SIZE, IMG_SIZE))
blob = blob.astype("float32") / 255

sigma = math.sqrt(np.sum((blob - np.mean(blob))**2) / blob.size)
blob = (blob - np.mean(blob)) / sigma

return blob

# Forward for Tensorflow
img = normalize(img)
tf_predictions = tf_model(img).numpy()

# Forward for OpenCV
img = normalize(img)
cv_model.setInput(img.reshape(1, 1, IMG_SIZE, IMG_SIZE))
cv_predictions = model.forward()

# Output results
print("OpenCV:", cv_predictions)
print("TF    :", tf_predictions)


It works great and OpenCV brings 10 times better performance! Just amazing!

But comparing the prediction results between Tensorflow (not freezed) model with OpenCV (freezed and optimized) model I found, that a large percentage (5% or 10%) of predictions differs by a lot. Some predictions even flips from Yes to No. Like an example here

OpenCV: [[0.4270926  0.57290745]]
TF    : [[0.9662978  0.03370225]]


My questions:

• What is the reason for this inaccuracy?

• Can I get my accuracy back by losing some of the amazing performance?

• May it be the precision drop from float32 to float16 and if yes, how can I force OpenCV to use float32?

### OpenCV predictions much worse compared to Tensorflow

I have a ResNet type of model, that I have simplified and that I would like to use with OpenCV.

Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
input_1 (InputLayer)            [(None, 50, 50, 1)]  0
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 50, 50, 1)    4           input_1[0][0]
__________________________________________________________________________________________________
conv2d (Conv2D)                 (None, 50, 50, 20)   200         batch_normalization[0][0]
__________________________________________________________________________________________________
re_lu (ReLU)                    (None, 50, 50, 20)   0           conv2d[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 50, 50, 20)   80          re_lu[0][0]
__________________________________________________________________________________________________

...

conv2d_13 (Conv2D)              (None, 13, 13, 80)   57680       batch_normalization_12[0][0]
__________________________________________________________________________________________________
conv2d_13[0][0]
__________________________________________________________________________________________________
re_lu_12 (ReLU)                 (None, 13, 13, 80)   0           add_5[0][0]
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, 13, 13, 80)   320         re_lu_12[0][0]
__________________________________________________________________________________________________
average_pooling2d (AveragePooli (None, 3, 3, 80)     0           batch_normalization_13[0][0]
__________________________________________________________________________________________________
flatten (Flatten)               (None, 720)          0           average_pooling2d[0][0]
__________________________________________________________________________________________________
dense (Dense)                   (None, 2)            1442        flatten[0][0]
==================================================================================================


I followed the tutorial to freeze the model and to optimize it using Tensorflow 1.5 tensorflow.python.tools.optimize_for_inference tool.

• https://medium.com/@sebastingarcaacosta/how-to-export-a-tensorflow-2-x-keras-model-to-a-frozen-and-optimized-graph-39740846d9eb

My code:

# Load Tensorflow model

# Normalize image
IMG_SIZE = 50
def normalize(img: np.ndarray) -> np.ndarray:
blob = cv2.resize(img, (IMG_SIZE, IMG_SIZE))
blob = blob.astype("float32") / 255

sigma = math.sqrt(np.sum((blob - np.mean(blob))**2) / blob.size)
blob = (blob - np.mean(blob)) / sigma

return blob

# Forward for Tensorflow
img = normalize(img)
tf_predictions = tf_model(img).numpy()

# Forward for OpenCV
img = normalize(img)
cv_model.setInput(img.reshape(1, 1, IMG_SIZE, IMG_SIZE))
cv_predictions = model.forward()

# Output results
print("OpenCV:", cv_predictions)
print("TF    :", tf_predictions)


It works great and OpenCV brings 10 times better performance! Just amazing!

But comparing the prediction results between Tensorflow (not freezed) model with OpenCV (freezed and optimized) model I found, that a large percentage (5% or 10%) of predictions differs by a lot. Some predictions even flips from Yes to No. Like an example here

OpenCV: [[0.4270926  0.57290745]]
TF    : [[0.9662978  0.03370225]]


My questions:

• What is the reason for this inaccuracy?

• Can I get my accuracy back by losing some of the amazing performance?

• May it be the precision drop from float32 to float16 and if yes, how can I force OpenCV to use float32?

Infos:

• Tensorflow: 2.3.0 (CPU only version)
• OpenCV: 4.4.0.44
• Python: 3.6.9