Ask Your Question

MennoK's profile - activity

2019-06-14 13:50:30 -0500 received badge  Popular Question (source)
2019-01-06 11:22:43 -0500 received badge  Student (source)
2018-02-15 09:45:05 -0500 commented question How to import Tensorflow's MobileNet into Opencv dnn?

@dkurt After 3 days of trying, I'm still not able to build the graph transformation tool on Windows 10. Do you have any

2018-02-13 02:08:13 -0500 received badge  Enthusiast
2018-02-12 04:33:46 -0500 commented question How to import Tensorflow's MobileNet into Opencv dnn?

Here is a link to the output_graph.pb: https://drive.google.com/file/d/1F-p-pTrd8K3rEts0zdQIC3rms97myQyb/view?usp=sharin

2018-02-12 04:32:27 -0500 asked a question How to import Tensorflow's MobileNet into Opencv dnn?

How to import Tensorflow's MobileNet into Opencv dnn? Hi, I'm retraining Mobilenet using tensorflow's retrain.py script

2018-02-05 04:05:09 -0500 marked best answer Opencv dnn import dropout layer error after finetuning Keras vgg16

Hi,

A few days ago I asked a question about importing a pretrained keras vgg16 model into Opencv dnn [1].

Now I finetuned the vgg16 for my own application by excluding the existed imagenet head and adding a new head to the model. Below shows the "pseudocode" how it's done:

    baseModel = VGG16(input_shape=(224, 224, 3), weights='imagenet', include_top=False)

    headModel = baseModel.output

    headModel = Flatten(name="flatten")(headModel)
    headModel = Dense(256, activation="relu")(headModel)
    headModel = Dropout(0.5)(headModel)
    headModel = Dense(5, activation="softmax")(headModel)

    model = Model(inputs=baseModel.input, outputs=headModel)

Subsequently, I train the new model with my own data and export it similar to the answer of my previous question. However when I try to read the net into opencv, it returns a ImportError:

 cv2.error: C:\projects\opencv-python\opencv\modules\dnn\src\tensorflow\tf_importer.cpp:1487: error: (-2) Unknown layer type PlaceholderWithDefault in op dropout_1/keras_learning_phase in function cv::dnn::experimental_dnn_v3::`anonymous-namespace'::TFImporter::populateNet

I've read on github, there is a solution to include dropout layers (https://github.com/opencv/opencv/pull...). Do you have any suggestions on how to implement this with keras? Or am I just making it myself difficult using Keras on top of tensorflow.

I have one more additional question: Do you ever plan to implement a readNetFromKeras(...) where a config.json and weights.h5 is given?

Edit:

Pbtxt file before (so flatten and dropout layers are included)

 ... some stuff before ...    
 node {
  name: "flatten/Reshape"
  op: "Reshape"
  input: "block5_pool/MaxPool"
  input: "flatten/stack"
}
node {
  name: "dense_1/MatMul"
  op: "MatMul"
  input: "flatten/Reshape"
  input: "dense_1/kernel"
  attr {
    key: "transpose_a"
    value {
      b: false
    }
  }
  attr {
    key: "transpose_b"
    value {
      b: false
    }
  }
}
node {
  name: "dense_1/BiasAdd"
  op: "BiasAdd"
  input: "dense_1/MatMul"
  input: "dense_1/bias"
}
node {
  name: "dense_1/Relu"
  op: "Relu"
  input: "dense_1/BiasAdd"
}
node {
  name: "dropout_1/keras_learning_phase"
  op: "PlaceholderWithDefault"
  input: "dropout_1/keras_learning_phase/input"
  attr {
    key: "dtype"
    value {
      type: DT_BOOL
    }
  }
  attr {
    key: "shape"
    value {
      shape {
      }
    }
  }
}
node {
  name: "dropout_1/cond/Switch"
  op: "Switch"
  input: "dropout_1/keras_learning_phase"
  input: "dropout_1/keras_learning_phase"
}
node {
  name: "dropout_1/cond/mul/Switch"
  op: "Switch"
  input: "dense_1/Relu"
  input: "dropout_1/keras_learning_phase"
  attr {
    key: "_class"
    value {
      list {
        s: "loc:@dense_1/Relu"
      }
    }
  }
}
node {
  name: "dropout_1/cond/mul"
  op: "Mul"
  input: "dropout_1/cond/mul/Switch:1"
  input: "dropout_1/cond/mul/y"
}
node {
  name: "dropout_1/cond/dropout/Shape"
  op: "Shape"
  input: "dropout_1/cond/mul"
  attr {
    key: "out_type"
    value {
      type: DT_INT32
    }
  }
}
node {
  name: "dropout_1/cond/dropout/random_uniform/RandomUniform"
  op: "RandomUniform"
  input: "dropout_1/cond/dropout/Shape"
  attr {
    key: "dtype"
    value {
      type: DT_FLOAT
    }
  }
  attr {
    key: "seed"
    value {
      i: 87654321
    }
  }
  attr {
    key: "seed2"
    value {
      i: 7788661
    }
  }
}
node {
  name: "dropout_1/cond/dropout/random_uniform/sub"
  op: "Sub"
  input: "dropout_1/cond/dropout/random_uniform/max"
  input: "dropout_1/cond/dropout/random_uniform/min"
}
node {
  name: "dropout_1/cond/dropout/random_uniform/mul"
  op: "Mul"
  input: "dropout_1/cond/dropout/random_uniform/RandomUniform"
  input: "dropout_1/cond/dropout/random_uniform/sub"
}
node {
  name: "dropout_1/cond/dropout/random_uniform"
  op: "Add"
  input: "dropout_1/cond/dropout/random_uniform/mul"
  input: "dropout_1/cond/dropout/random_uniform/min"
}
node {
  name: "dropout_1/cond/dropout/add"
  op: "Add"
  input: "dropout_1/cond/dropout/keep_prob"
  input: "dropout_1 ...
(more)
2018-02-05 04:05:08 -0500 received badge  Supporter (source)
2018-02-05 04:03:49 -0500 commented answer Opencv dnn import dropout layer error after finetuning Keras vgg16

The float32 fixed everything. Opencv is working fine now and I receive the expected results now. Thanks a lot :)

2018-02-05 03:48:30 -0500 commented answer Opencv dnn import dropout layer error after finetuning Keras vgg16

@dkurt I tried your suggestions: 1) use the result image of blobFromImage both for tensorflow (transposing to (0,2,3,1)

2018-02-05 03:48:01 -0500 commented answer Opencv dnn import dropout layer error after finetuning Keras vgg16

@dkurt I tried your suggestions: 1) use result of blobFromImage both for tensorflow (transposing to (0,2,3,1) and opencv

2018-02-02 09:40:01 -0500 commented answer Opencv dnn import dropout layer error after finetuning Keras vgg16

True, Ill try to figure this out. Thanks for the responses and good luck with the further development of dnn

2018-02-02 06:28:18 -0500 commented answer Opencv dnn import dropout layer error after finetuning Keras vgg16

@dkurt you're indicating that an input image for tensorflow and opencv are different after the preprocessing is done?

2018-02-02 04:28:23 -0500 commented answer Opencv dnn import dropout layer error after finetuning Keras vgg16

Tensorflow code: # Read the graph. with tf.gfile.FastGFile('model.pb', 'rb') as f: graph_def = tf.GraphDef() gr

2018-02-02 04:27:03 -0500 commented answer Opencv dnn import dropout layer error after finetuning Keras vgg16

Tensorflow code: with tf.Session() as sess: # Restore session sess.graph.as_default() tf.import_graph_def(g

2018-02-02 04:26:32 -0500 commented answer Opencv dnn import dropout layer error after finetuning Keras vgg16

Using the code from the Test in your answer. The max absolute error is 1.937151e-07. Also the outputs are the same Outp

2018-02-02 04:26:15 -0500 commented answer Opencv dnn import dropout layer error after finetuning Keras vgg16

Tensorflow code: with tf.Session() as sess: # Restore session sess.graph.as_default() tf.import_graph_def(g

2018-02-02 04:22:18 -0500 commented answer Opencv dnn import dropout layer error after finetuning Keras vgg16

Using the code from the Test in your answer. The max absolute error is 1.937151e-07. Also the outputs are the same Outp

2018-02-02 04:21:21 -0500 commented answer Opencv dnn import dropout layer error after finetuning Keras vgg16

Using the code from the Test in your answer. The max absolute error is 1.937151e-07. Also the outputs are the same Outp

2018-02-02 02:52:53 -0500 commented answer Opencv dnn import dropout layer error after finetuning Keras vgg16

Results Tensorflow: [[1846 143 0 0 3] [ 4 281 18 0 1] [ 0 14 183 0 2] [ 0 0

2018-02-02 02:48:50 -0500 commented answer Opencv dnn import dropout layer error after finetuning Keras vgg16

GetLabels is own function to retrieve label indices from the imagepaths

2018-02-02 02:46:26 -0500 commented answer Opencv dnn import dropout layer error after finetuning Keras vgg16

@dkurt thank you for your responses. I tested with tensorflow, and the results were as expected (same as in Keras). Howe

2018-02-01 08:56:09 -0500 commented question Opencv dnn import dropout layer error after finetuning Keras vgg16

@dkurt I'm not sure I understand what you mean. I can't send all the code, but I'm sure I receive the expected results i

2018-02-01 07:33:18 -0500 commented question Opencv dnn import dropout layer error after finetuning Keras vgg16

After this step I followed your instructions

2018-02-01 07:32:17 -0500 commented question Opencv dnn import dropout layer error after finetuning Keras vgg16

Code to convert Keras to Tensorflow: jsonFile = open('vgg16.json', 'r') loadModelJson = jsonFile.read() model = model_f

2018-02-01 07:32:05 -0500 commented question Opencv dnn import dropout layer error after finetuning Keras vgg16

Code to convert Keras to Tensorflow: jsonFile = open('modelweights_vgg16.json', 'r') loadModelJson = jsonFile.read() mo

2018-02-01 07:31:24 -0500 commented question Opencv dnn import dropout layer error after finetuning Keras vgg16

jsonFile = open('modelweights_vgg16.json', 'r') loadModelJson = jsonFile.read() model = model_from_json(loadMode

2018-02-01 07:31:05 -0500 commented question Opencv dnn import dropout layer error after finetuning Keras vgg16

jsonFile = open('vgg16.json', 'r') loadModelJson = jsonFile.read() model = model_from_json(loadModelJson) mo

2018-02-01 07:30:32 -0500 commented question Opencv dnn import dropout layer error after finetuning Keras vgg16

jsonFile = open('vgg16.json', 'r') loadModelJson = jsonFile.read() model = model_from_json(loadModelJson

2018-02-01 07:30:18 -0500 commented question Opencv dnn import dropout layer error after finetuning Keras vgg16

jsonFile = open('vgg16.json', 'r') loadModelJson = jsonFile.read() model = model_from_json(loadModelJson) mo

2018-02-01 07:30:07 -0500 commented question Opencv dnn import dropout layer error after finetuning Keras vgg16

jsonFile = open('vgg16.json', 'r') loadModelJson = jsonFile.read() model = model_from_json(loadModelJson) mo

2018-02-01 07:29:55 -0500 commented question Opencv dnn import dropout layer error after finetuning Keras vgg16

jsonFile = open('modelweights_vgg16.json', 'r') loadModelJson = jsonFile.read() model = model_from_json(loadMode

2018-02-01 07:29:49 -0500 commented question Opencv dnn import dropout layer error after finetuning Keras vgg16

imports jsonFile = open('modelweights_vgg16.json', 'r') loadModelJson = jsonFile.read() model = model_from_json(loadMod

2018-02-01 07:29:41 -0500 commented question Opencv dnn import dropout layer error after finetuning Keras vgg16

imports jsonFile = open('modelweights_vgg16.json', 'r') loadModelJson = jsonFile.read() model = model_from_json(loadMod

2018-02-01 07:29:21 -0500 commented question Opencv dnn import dropout layer error after finetuning Keras vgg16

I use Keras on top of Tensorflow, so I test the results using Keras which is simply model.predict(image) for each image

2018-02-01 07:28:55 -0500 commented question Opencv dnn import dropout layer error after finetuning Keras vgg16

I use Keras on top of Tensorflow, so I test the results using Keras which is simply model.predict(image) for each image

2018-02-01 06:44:34 -0500 commented question Opencv dnn import dropout layer error after finetuning Keras vgg16

@dkurt I added a snippet of the .pbtxt file before and after the changes.

2018-02-01 06:43:43 -0500 edited question Opencv dnn import dropout layer error after finetuning Keras vgg16

Opencv dnn import dropout layer error after finetuning Keras vgg16 Hi, A few days ago I asked a question about importin

2018-02-01 05:49:02 -0500 commented question Opencv dnn import dropout layer error after finetuning Keras vgg16

Hey, I tried this already (if I remember correctly, you mentioned it in another github issue). So I removed every node w

2018-02-01 05:32:53 -0500 commented question Opencv dnn import dropout layer error after finetuning Keras vgg16

Hey, I tried this already (if I remember correctly, you mentioned it in another github issue). So I removed every node w

2018-02-01 04:32:04 -0500 edited question Opencv dnn import dropout layer error after finetuning Keras vgg16

Opencv dnn import dropout layer error after finetuning Keras vgg16 Hi, A few days ago I asked a question about importin

2018-02-01 04:23:35 -0500 commented question Opencv dnn import dropout layer error after finetuning Keras vgg16

Here is link to the used .pb and .pbtxt file: https://drive.google.com/file/d/120G3pC5HTODj9N5zM8wZXVEA2O_G2MYQ/view?usp

2018-02-01 04:20:20 -0500 asked a question Opencv dnn import dropout layer error after finetuning Keras vgg16

Opencv dnn import dropout layer error after finetuning Keras vgg16 Hi, A few days ago I asked a question about importin

2018-01-30 07:50:25 -0500 marked best answer Opencv dnn import error for keras pretrained Vgg16 model

Hi,

I'm trying to export the keras pretrained VGG16 model to a tensorflow model and then I want to import the tensorflow model into opencv. To do this, I got the following python code:

from keras import applications
from keras import backend as K
import cv2 as cv
import tensorflow as tf
from tensorflow.python.framework import graph_util
from tensorflow.python.framework import graph_io

model = applications.VGG16(input_shape=(224, 224, 3), weights='imagenet', include_top=True)
K.set_learning_phase(0)

pred_node_names = [None]
pred = [None]
for i in range(1):
    pred_node_names[i] = "output_node"+str(i)
    pred[i] = tf.identity(model.outputs[i], name=pred_node_names[i])

 sess = K.get_session()
 constant_graph = graph_util.convert_variables_to_constants(sess, sess.graph.as_graph_def(), pred_node_names)
 graph_io.write_graph(constant_graph, ".", "model.pb", as_text=False)

net = cv.dnn.readNetFromTensorflow('model.pb')

Executing this code, gives following error (similar in C++):

cv2.error: C:\projects\opencv-python\opencv\modules\dnn\src\tensorflow\tf_importer.cpp:1487: error: (-2) Unknown layer type Shape in op flatten/Shape in function cv::dnn::experimental_dnn_v3::`anonymous-namespace'::TFImporter::populateNet

When I'm not including the top, I don't receive this error message.

Do you have any suggestions?

2018-01-30 07:50:25 -0500 received badge  Scholar (source)
2018-01-30 07:50:17 -0500 commented answer Opencv dnn import error for keras pretrained Vgg16 model

Thanks a lot again. Now, I'm also able to import the model in open cv C++