Getting inaccurate results using tensorflow net and opencv dnn

asked 2018-03-26 04:14:34 -0600

dan01 gravatar image

updated 2018-03-26 04:18:57 -0600

I'm trying to use a tensorflow frozen graph to detect hands in an image. I get some weird extra boxes when I'm loading the net with opencv.

I am using the frozen graph from this handtracking github repo, for which I am generating the text graph using tf_text_graph_ssd.py

Result from Tensorflow: image description

Result from OpenCV: image description

The code and files are here: Github

System configuration:

  • Ubuntu 16.04
  • Python 2.7
  • OpenCV 3.4.1
  • Tensorflow 1.6.0

The only difference between the two methods that I can think about is the graph generation. Tensorflow loads directly the *.pb file, whereas opencv needs the *.pbtxt. Maybe the tool generating the text graph isn't completely accurate? Any suggestions are welcome.

edit retag flag offensive close merge delete

Comments

@dan01, May I ask you to compare your case with the following one? http://answers.opencv.org/question/18... . Did you use a default *.cofigfile (https://github.com/tensorflow/models/...) with modified number of classes or you changed min_scale / max_scale? Your case is very useful to fix it. Thank you!

dkurt gravatar imagedkurt ( 2018-03-26 04:27:52 -0600 )edit
1

Hi @dkurt, thanks for the fast reply. I didn't train the model myself, I just wanted to use it, so I actually didn't use the config file, just the frozen graph. The config file is from here

So here's the diff between the original config and the one on the tensorflow github out.diff

TL;DR:

  • num_classes: 1
  • anchorwise_output: true - for classification_loss
  • batch_size: 6 - in train config
  • num_examples: 960 - in eval config
  • the paths to model ckpt, train and test records and pbtxt
dan01 gravatar imagedan01 ( 2018-03-26 04:58:36 -0600 )edit

@dan01, Thank you! The model is correct. All that you need is just use higher confidence threshold (0.2 is now).

dkurt gravatar imagedkurt ( 2018-03-26 05:31:13 -0600 )edit

Hmm, I don't think it's correct... I've printed the scores and they differ from one model to another (updated images in original post), which led me to try another image. Now i get results similar to this post. Please see the image

wrong boxes.

P.S.: Tensorflow method yields correct results, see here.

dan01 gravatar imagedan01 ( 2018-03-26 06:50:21 -0600 )edit

@dkurt, setting it to if score > 0.8: yields

image description

the correct box has a score of only 0.26, here's a higher resolution image.

dan01 gravatar imagedan01 ( 2018-03-26 07:04:57 -0600 )edit

@dan01, besides trying to increase a confidence threshold from 0.2 to 0.6 (if score > 0.2: -> if score > 0.6:), may I ask you to test the first image again but with .png image format but not .jpg? You may convert it using OpenCV's cv.imwrite('/path/to/png', cv.imread('/path/to/jpg')).

dkurt gravatar imagedkurt ( 2018-03-26 07:06:39 -0600 )edit

@dkurt, done, but the results are the same. Please see below:

if score > 0.2:result1

if score > 0.6:result2

dan01 gravatar imagedan01 ( 2018-03-26 07:23:35 -0600 )edit
1

@dan01, I got the problem. This is a deprecated SSD graph since TensorFlow 1.4. You need to modify an every PriorBox_* layer (except PriorBox_0) in the following way: in width and height attribute move the last float_val to be the second one (float_val[0], float_val[1], ..., float_val[N-1] - > float_val[0], float_val[N-1], float_val[1], ..., float_val[N-2]). So it reverts the thing mentioned here.

dkurt gravatar imagedkurt ( 2018-03-26 07:41:55 -0600 )edit

@dkurt Wow that fixed it! Thank you! So if I re-train the model, this problem shouldn't come up?

dan01 gravatar imagedan01 ( 2018-03-26 07:47:05 -0600 )edit
1

@dan01, It's not necessary if you use TensorFlow >= 1.4 and train the model from scratch. However if you wanted to tune this one I think you need to do the same again.

dkurt gravatar imagedkurt ( 2018-03-26 07:51:36 -0600 )edit