Ask Your Question

Issues running dnn openpose sample code with kinect v2

asked 2019-01-04 22:37:38 -0500

Ed95 gravatar image

updated 2019-01-05 02:05:30 -0500

Hi, I have version 4.0.1-dev installed on ubuntu 16.04 (g++ v 5.4.0) with libfreenect2 (for kinect v2) and am trying to use the dnn openpose sample program with a kinect camera. The sample code works with a single image. The kinect driver can (independently) display an rgb stream using imshow(). The issue I'm having is using them both together. **I have assumed the sample code can be used with a web camera (kinect) (instead of just still images). The code I have throws a CV:Exception when it calls net.forward().

Mat inputBlob = blobFromImage(img, scale, Size(W_in, H_in), Scalar(0, 0, 0), false, false, CV_8U);
    Mat result = net.forward();

The exception is :

   terminate called after throwing an instance of 'cv::Exception'
      what():  OpenCV(4.0.1-dev) /home/ed/src/opencv/modules/dnn/src/layers/convolution_layer.cpp:1119: error: (-215:Assertion failed) inputs[0].size[1] % blobs[0].size[1] == 0 in function 'forward'

It suggests the inputBlob maybe empty. The output from gdb is below:

(gdb) print inputBlob
$2 = {flags = 1124024320, dims = 4, rows = -1, cols = -1, data = 0x114acb00 "hfhifc^Y[[[XUXYUTPMLOQK?79877:?GPVULC>9<@A?>?C:76:Fc\243\260\253\205aSNF??>EJIA:5-.47<A>94/41/-)--+0D\214\226\220R.())&$#!$-64+&$&*+-/12221//268:8579>>::==:9746??@GJC]\233\304\301\250\226\245\323\340\336\331\333\334\334\340\336\333\333\334\337\337\337\336\336\336\336\336\334\335\337\340\340\340\343\343\342\342\344\344\341\343\343\341\337\340\343\345\344\342\337\337\342\345\344", <incomplete sequence \342>..., datastart = 0x114acb00 "hfhifc^Y[[[XUXYUTPMLOQK?79877:?GPVULC>9<@A?>?C:76:Fc\243\260\253\205aSNF??>EJIA:5-.47<A>94/41/-)--+0D\214\226\220R.())&$#!$-64+&$&*+-/12221//268:8579>>::==:9746??@GJC]\233\304\301\250\226\245\323\340\336\331\333\334\334\340\336\333\333\334\337\337\337\336\336\336\336\336\334\335\337\340\340\340\343\343\342\342\344\344\341\343\343\341\337\340\343\345\344\342\337\337\342\345\344", <incomplete sequence \342>..., dataend = 0x11c95b00 "", datalimit = 0x11c95b00 "", allocator = 0x0, u = 0xe8b860, size = {p = 0xeccae4}, step = {p = 0xeccac0, buf = {0, 0}}}

So I think the issues could be :

1) How I create the cv::Mat for the kinect rgb image.

   cv::Mat rgbCV(rgb->height,rgb->width,CV_8UC4,rgb->data);
   cv::Mat img(rgbCV.clone());

2)Creating the inputBlob

 Mat inputBlob = blobFromImage(img, scale, Size(W_in, H_in), Scalar(0, 0, 0), false, false, CV_8U);

W_in is set to 1920, H_in is set to 1080 (kinect rgb size), scale =1.0. Initially I had the type (last parameter) on default (CV_32F) but tried CV_8U. The result was the same cv.Exception).

All help is appreciated.

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted

answered 2019-01-05 00:32:33 -0500

berak gravatar image

updated 2019-01-05 10:37:21 -0500

indeed, your Mat constructor:

cv::Mat rgbCV(rgb->height,rgb->width,CV_8UC4,rgb->data);

is not suitable for the dnn input, which requires CV_8UC3 (bgr vs bgra).

you have to convert it:

Mat img;
cvtColor(rgbCV, img, COLOR_BGRA2BGR);

before you can feed it into the dnn

also, the rows and cols you see in your debugger are bogus. a 2d Size cannot hold the 4 dimensions used for the dnn blobs (and thus they're set to set to -1), therefore you have to inspect the size member, like this:

mat.size[0]  // num images in the blob
mat.size[1]  // num channels (or planes)
mat.size[2]  // height
mat.size[3]  // width
edit flag offensive delete link more


That's fixed it. Thanks! Appreciate the debugger hint. That will be useful.

Ed95 gravatar imageEd95 ( 2019-01-05 02:25:35 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower


Asked: 2019-01-04 22:37:38 -0500

Seen: 38 times

Last updated: Jan 05