readNetFromTensorflow() errors from Mask_RCNN model

asked 2019-01-08 01:55:49 -0600

lunasdejavu gravatar image

updated 2019-01-08 02:00:06 -0600

I made my Mask_RCNN model from this github project it is a project written with tensorflow and keras. Enviroment : win7 x64 visual studio 2015 opencv 4.0.1 tensorflow 1.12 GPU gtx1060 CUDA 9.0 since it saves its weights to .h5 file, I want to turn it to .pb and .pbtxt so that I can read it by readNetFromTensorflow(). I wrote something like

def h5_to_pb(h5_model,output_dir,model_name,out_prefix = "output_",log_tensorboard = True):
    if osp.exists(output_dir) == False:
        os.mkdir(output_dir)
    out_nodes = []
    for i in range(len(h5_model.outputs)):
        out_nodes.append(out_prefix + str(i + 1))
        tf.identity(h5_model.output[i],out_prefix + str(i + 1))
    sess = K.get_session()
    from tensorflow.python.framework import graph_util,graph_io
    init_graph = sess.graph.as_graph_def()
    main_graph = graph_util.convert_variables_to_constants(sess,init_graph,out_nodes)
    graph_io.write_graph(main_graph,output_dir,name = model_name,as_text = False)
    if log_tensorboard:
        from tensorflow.python.tools import import_pb_to_tensorboard
        import_pb_to_tensorboard.import_to_tensorboard(osp.join(output_dir,model_name),output_dir)

output_dir = osp.join(os.getcwd(),"trans_model")
output_dir = "D:/"

ROOT_DIR = os.path.abspath("C:/Mask_RCNN/")
MODEL_DIR = os.path.join(ROOT_DIR, "logs")
model = modellib.MaskRCNN(mode="inference", config=config,
                      model_dir=MODEL_DIR)
# model.keras_model.summary()
# print(model.keras_model.inputs)
# print(model.keras_model.outputs)

model.load_weights(weight_file_path, by_name=True)
h5_model = model.keras_model
print(len(h5_model.outputs))
h5_to_pb(h5_model,output_dir = output_dir,model_name = output_graph_name)

then I turned pb to pbtxt

def convert_pb_to_pbtxt(filename):
  with gfile.FastGFile(filename,'rb') as f:
    graph_def = tf.GraphDef()

    graph_def.ParseFromString(f.read())

    tf.import_graph_def(graph_def, name='')

    tf.train.write_graph(graph_def, './', 'protobuf.pbtxt', as_text=True)
  return

then I got the error Error parsing text-format opencv_te nsorflow.GraphDef: 41208:5: Unknown enumeration value of "DT_RESOURCE" for field "type". OpenCV(4.0.1) Error: Unspecified error (FAILED: ReadProtoFromTextFile(param_file , param). Failed to parse GraphDef file: D:/model_1.pbtxt) in cv::dnn::ReadTFNet ParamsFromTextFileOrDie, file C:\build\master_winpack-build-win64-vc14\opencv\mo dules\dnn\src\tensorflow\tf_io.cpp, line 54

it seem that the format of the pbtxt is not right, so can anyone tell me how to turn the model I trained to let the opencv read it to do the object detection please?

edit retag flag offensive close merge delete

Comments

i think, that your pbtxt conversion routine is too simple, and that you have to run this script on it instead.

berak gravatar imageberak ( 2019-01-08 02:01:21 -0600 )edit

also please have a look at the wiki page

berak gravatar imageberak ( 2019-01-08 02:07:00 -0600 )edit

thanks! since I only need the inference mode, should I mark everything about config?

lunasdejavu gravatar imagelunasdejavu ( 2019-01-08 02:38:04 -0600 )edit

should I mark everything about config?

what do you mean ?

oh, sorry, i begin to see the problem: there is no config file, it's all hardcoded in the keras scripts ;(

berak gravatar imageberak ( 2019-01-08 02:47:52 -0600 )edit
1

it said "Pass a configuration file which was used for training to help script determine hyper-parameters." but there must be a configuration file for tf_text_graph_mask_rcnn.py, so should I mark them all?

lunasdejavu gravatar imagelunasdejavu ( 2019-01-08 02:58:43 -0600 )edit

ya because it is ResNet 50 Mask_RCNN,there is no config from tensorflow.

lunasdejavu gravatar imagelunasdejavu ( 2019-01-08 03:02:12 -0600 )edit
1

i still don't know, what mark them all means. mark what ? where ?

apart from that i don''t have a good idea, how to solve it now. @dkurt , -- we need your help ;)

berak gravatar imageberak ( 2019-01-08 03:06:20 -0600 )edit
berak gravatar imageberak ( 2019-01-08 03:08:19 -0600 )edit

thanks, I am already reading it lol. I said mark just because the wiki said configuration file was used for training since I don't want to train it, I might not really need the configuration file.

lunasdejavu gravatar imagelunasdejavu ( 2019-01-08 03:14:31 -0600 )edit

i'm having trouble running your scripts above (on colab). imports are missing, the config is undefined, etc.

berak gravatar imageberak ( 2019-01-08 03:49:06 -0600 )edit
1

The thing is that currently OpenCV supports only Mask-RCNN network from TensorFlow Object Detection API (see the guidementioned by @berak). It's implementation of Mask-RCNN differs from one in https://github.com/matterport/Mask_RCNN/. Configuration file is a protobuf text message which is used in TF OD API and helps us to build an optional .pbtxt with network topology.

dkurt gravatar imagedkurt ( 2019-01-08 04:54:24 -0600 )edit

wait so if I'd like to use the opencv to do the Mask R_CNN ,I only can choose inception-net v2 as the backbone?

lunasdejavu gravatar imagelunasdejavu ( 2019-01-13 20:46:56 -0600 )edit

@lunasdejavu, According configuration file, you can vary it in feature_extractor field.

dkurt gravatar imagedkurt ( 2019-01-14 00:43:30 -0600 )edit