Ask Your Question
0

another tensorflow import problem

asked 2017-10-23 04:13:44 -0500

berak gravatar image

updated 2017-10-23 04:22:09 -0500

a fairly simple network crashes the tf_importer:

OpenCV Error: Assertion failed (!beginsData.empty()) in populateNet, file C:\p\opencv\modules\dnn\src\tensorflow\tf_importer.cpp, line 941

(full code here)

def conv_net(x, n_classes, dropout, reuse, is_training):
    # Define a scope for reusing the variables
    with tf.variable_scope('ConvNet', reuse=reuse):

        # Convolution Layer with 32 filters and a kernel size of 5
        conv1 = tf.layers.conv2d(x, 32, 5, activation=tf.nn.relu)
        # Max Pooling (down-sampling) with strides of 2 and kernel size of 2
        conv1 = tf.layers.max_pooling2d(conv1, 2, 2)

        # Convolution Layer with 32 filters and a kernel size of 5
        conv2 = tf.layers.conv2d(conv1, 64, 3, activation=tf.nn.relu)
        # Max Pooling (down-sampling) with strides of 2 and kernel size of 2
        conv2 = tf.layers.max_pooling2d(conv2, 2, 2)

        # Flatten the data to a 1-D vector for the fully connected layer
        fc1 = tf.contrib.layers.flatten(conv2)

        # Fully connected layer (in contrib folder for now)
        fc1 = tf.layers.dense(fc1, 1024)
        # Apply Dropout (if is_training is False, dropout is not applied)
        fc1 = tf.layers.dropout(fc1, rate=dropout, training=is_training)

        # Output layer, class prediction
        out = tf.layers.dense(fc1, n_classes)
        # Because 'softmax_cross_entropy_with_logits' already apply softmax,
        # we only apply softmax to testing network
        out = tf.nn.softmax(out) if not is_training else out

    return out

then we have:

 python freeze.py --output_node ConvNet/Softmax --model_dir .
 python optimize_for_inference.py --input frozen_model.pb --output face_opt.pb --frozen_graph True --input_names ConvNet_1/conv2d/convolution  --output_names ConvNet_1/Softmax

and some minimal c++ code, to load it into the dnn:

int main(int argc, char **argv)
{
    String modelBin = "mnist/face_opt.pb";
    String imageFile = (argc > 1) ? argv[1] : "test.jpg";
    Net net = dnn::readNetFromTensorflow(modelBin);
    Mat img = imread(imageFile, 0);
    resize(img, img, Size(64,64));  
    Mat inputBlob = blobFromImage(img);
    net.setInput(inputBlob, "ConvNet_1/conv2d/convolution"); 

    Mat probMat =  net.forward();
    Point p; minMaxLoc(probMat, 0,0,0, &p);
    cout << p.x << " " << probMat << endl;
    return 0;
}

resulting in a crash:

OpenCV Error: Assertion failed (!beginsData.empty()) in populateNet, file C:\p\opencv\modules\dnn\src\tensorflow\tf_importer.cpp, line 941

debugging it a bit, i see, that tf inserted some layers here:

    at C:\p\opencv\modules\dnn\src\tensorflow\tf_importer.cpp:623
623             LayerParams layerParams;
(gdb) p name.c_str()
$1 = 0x5b21644 "ConvNet/conv2d/bias"
...
$2 = 0x5b21684 "ConvNet/conv2d_1/kernel"
$3 = 0x5b216c4 "ConvNet/conv2d_1/bias"
$4 = 0x5b215c4 "ConvNet/dense/kernel"
$5 = 0x5b21584 "ConvNet/dense/bias"
$6 = 0x5b215c4 "ConvNet/dense_1/kernel"
$7 = 0x5b21644 "ConvNet/dense_1/bias"
$8 = 0x5b21244 "ConvNet_1/conv2d/convolution"
$9 = 0x5b21684 "ConvNet_1/conv2d/BiasAdd"
$10 = 0x5b214c4 "ConvNet_1/conv2d/Relu"
$11 = 0x5b6c7c4 "ConvNet_1/max_pooling2d/MaxPool"
$12 = 0x5b6c304 "ConvNet_1/conv2d_2/convolution"
$13 = 0x5b6cb04 "ConvNet_1/conv2d_2/BiasAdd"
$14 = 0x5b6d2c4 "ConvNet_1/conv2d_2/Relu"
$15 = 0x5b21104 "ConvNet_1/max_pooling2d_2/MaxPool"
$16 = 0x5b6d6c4 "ConvNet_1/Flatten/Shape"
$17 = 0x5b6ec44 "ConvNet_1/Flatten/Slice/begin"
$18 = 0x5b6ec44 "ConvNet_1/Flatten/Slice/size"
$19 = 0x5b6ed84 "ConvNet_1/Flatten/Slice"

looking here: i see, that beginsData is indeed an empty string, while sizesData seems to contain something.

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
1

answered 2017-10-25 08:48:21 -0500

dkurt gravatar image

updated 2017-10-25 08:49:10 -0500

Hi, @berak, DNN doesn't support TensorFlow's flatten op because it computes Shape of input in runtime. Then it does a reshape. There are several ways to make it more simple for DNN:

1. Use reshape op instead but compute input's shape out of the graph:

total = int(np.prod(inp.shape[1:]))
flattened = tf.reshape(inp, [-1, total])

2. More preferable way because it'll solve both flatten and dropout ops import:

2.1. Freeze and optimize graph as you did.

2.2. Call the following script to create a text graph representation:

import tensorflow as tf

# Read the graph.
with tf.gfile.FastGFile('face_opt.pb') as f:
    graph_def = tf.GraphDef()
    graph_def.ParseFromString(f.read())

# Remove Const nodes.
for i in reversed(range(len(graph_def.node))):
    if graph_def.node[i].op == 'Const':
        del graph_def.node[i]
    for attr in ['T', 'data_format', 'Tshape', 'N', 'Tidx', 'Tdim',
                 'use_cudnn_on_gpu', 'Index', 'Tperm', 'is_training',
                 'Tpaddings']:
        if attr in graph_def.node[i].attr:
            del graph_def.node[i].attr[attr]

# Save as text.
tf.train.write_graph(graph_def, "", "text_graph.pbtxt", as_text=True)

2.3. Replace a subgraph of nodes ConvNet/Flatten/Shape, ConvNet/Flatten/Slice, ConvNet/Flatten/Slice_1, ConvNet/Flatten/Prod, ConvNet/Flatten/ExpandDim, ConvNet/Flatten/concat, ConvNet/Flatten/Reshape onto the following node:

node {
  name: "ConvNet/Flatten/Reshape"
  op: "Flatten"
  input: "ConvNet/max_pooling2d_2/MaxPool"
}

2.4. Remove a subgraph from ConvNet/dropout/dropout/Shape to ConvNet/dropout/dropout/mul (inclusive both). Replace ConvNet/dense_2/MatMul's input from ConvNet/dropout/dropout/mul to ConvNet/dense/BiasAdd.

Then use both binary graph and a text one during import: https://docs.opencv.org/master/d6/d0f... .

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-10-23 04:13:44 -0500

Seen: 661 times

Last updated: Oct 25 '17