Multi-person poses' detection using OpenCV + OpenPose

asked 2018-04-09 09:30:23 -0600

updated 2018-04-10 05:49:42 -0600

Hello, dear colleagiues!

I recently started studying computer vision. I'm very interested this direction. Now I'm studying the recognition of human poses, and I'm trying to remake some examples in C # to experiment further in Unity.

However, I can not find documentation or examples of how I could improve my code anywhere to find the poses of all people in the image. Could you tell me how I should change my code so that I can recognize the poses for all people in the image?

CODE FRAGMENT:

private void RecognizePose( Mat image, Mat output_image, List<point> points, float[] data ) {

for( int i = 0; i < BODY_PARTS_MPI.Count; i++ ) {

    output_image.get( i, 0, data );

    Mat heat_map = new Mat ( 1, data.Length, CvType.CV_32FC1 );

    heat_map.put( 0, 0, data );

    //Originally, we try to find all the local maximums. To simplify a sample
    //we just find a global one. However only a single pose at the same time
    //could be detected this way.
    Core.MinMaxLocResult result = Core.minMaxLoc( heat_map );

    heat_map.Dispose();

    double x = (

        (image.cols() * 
        (result.maxLoc.x % matrix_output_columns)) / 
        matrix_output_columns
    );

    double y = (

        (image.rows() * 
        (result.maxLoc.x / matrix_output_rows)) / 
        matrix_output_rows
    );

    if( result.maxVal > 0.1d ) points.Add( new Point( x, y ) );
    else points.Add( null );
}

}

Line "Core.MinMaxLocResult result = Core.minMaxLoc( heat_map );" and next processing gets us only single person's pose. And comment for this line says the same. But how can I get the all local maximumns and detect all poses?

With best regards and best wishes, Sergey Netyagin

edit retag flag offensive close merge delete

Comments

does this (java?) code even work for a single person ?

i'm having some doubts, if output_image.get( i, 0, data ); is retrieving the right thing, imho, there should be a output_image=output_image.reshape(1,19); before trying to access it's data

berak ( 2018-04-09 12:06:32 -0600 )edit

you'll HAVE TO read the paper
it needs the original, linevec.prototxt,not opencv's shortened one (it does not have access to the PAF's)
i made an attempt at it lately, but there are still bugs

berak ( 2018-04-10 06:20:07 -0600 )edit

Yes..!! It works perfectly fine for a single person.

The code is available at this link: https://github.com/opencv/opencv/blob...

The step to reproduce is as follows: python3 openpose.py --input=COCO_val2014_000000000589.jpg --proto=openpose_pose_coco.prototxt --model=pose_iter_440000.caffemodel --dataset=COCO

The data is available at:

--> COCO_val2014_000000000589.jpg

https://github.com/CMU-Perceptual-Com...

--> openpose_pose_coco.prototxt

https://github.com/opencv/opencv_extr...

--> pose_iter_440000.caffemodel

http://posefs1.perception.cs.cmu.edu/...

Gajjar_Mihir ( 2018-04-11 06:16:08 -0600 )edit

i know, i know.

i was only questioning your java version of it. (are you sure, you got the heatmap extraction right ?)

berak ( 2018-04-11 06:43:47 -0600 )edit

I am trying to implement it on android (java). The code works on ubuntu with opencv 3.4.1 but I am getting error on android with opencv 3.4.1. I have explained the error here: https://github.com/opencv/opencv/issu...

I tried the code on some images till now. It does fail sometimes.

Any help would be appreciated.

Gajjar_Mihir ( 2018-04-11 06:54:51 -0600 )edit

could you add that part of the code to your question ?

(btw, your github repo is no more available)

berak ( 2018-04-11 06:58:36 -0600 )edit

Should I upload the Android project on my Github repo or should I paste the code here?

I think uploading would be a better option because the entire code would be visible there with indentation.

Gajjar_Mihir ( 2018-04-11 08:31:49 -0600 )edit

a few lines with the result parsing here AND your whole code on github, maybe ?

berak ( 2018-04-11 08:58:44 -0600 )edit

int frameWidth = mat1.width();
int frameHeight = mat1.height();
Log.d("DEBUG",frameHeight + " " + frameWidth + " " + mat1);
Mat inp = Dnn.blobFromImage(mat1, 1, new Size(368, 368), new Scalar(0,0,0),false,false);
Log.d("See Hererere", inp.height() + " " + inp.width() + " " + inp.channels());
net.setInput(inp);
Mat out = net.forward();
Log.d("OUTPUT", out + "");

I am getting error in net.forward()

I have uploaded the entire code on github.

Gajjar_Mihir ( 2018-04-19 05:54:50 -0600 )edit

github link ?

maybe you can try this one in the meantime

berak ( 2018-04-19 06:07:55 -0600 )edit

The android project is about 700mb. Can we upload that on github?

Gajjar_Mihir ( 2018-04-19 06:19:57 -0600 )edit

see more comments

Multi-person poses' detection using OpenCV + OpenPose

Comments

1 answer

Comments

Links

Question Tools

Stats

Related questions

Multi-person poses' detection using OpenCV + OpenPose edit

Comments

1 answer

Comments

Links

Question Tools

Stats

Related questions

Multi-person poses' detection using OpenCV + OpenPose