Ask Your Question
2

Multi-person poses' detection using OpenCV + OpenPose

asked 2018-04-09 09:30:23 -0500

Sergey Netyagin gravatar image

updated 2018-04-10 05:49:42 -0500

Hello, dear colleagiues!

I recently started studying computer vision. I'm very interested this direction. Now I'm studying the recognition of human poses, and I'm trying to remake some examples in C # to experiment further in Unity.

However, I can not find documentation or examples of how I could improve my code anywhere to find the poses of all people in the image. Could you tell me how I should change my code so that I can recognize the poses for all people in the image?

CODE FRAGMENT:

private void RecognizePose( Mat image, Mat output_image, List<point> points, float[] data ) {

for( int i = 0; i < BODY_PARTS_MPI.Count; i++ ) {

    output_image.get( i, 0, data );

    Mat heat_map = new Mat ( 1, data.Length, CvType.CV_32FC1 );

    heat_map.put( 0, 0, data );

    //Originally, we try to find all the local maximums. To simplify a sample
    //we just find a global one. However only a single pose at the same time
    //could be detected this way.
    Core.MinMaxLocResult result = Core.minMaxLoc( heat_map );

    heat_map.Dispose();

    double x = (

        (image.cols() * 
        (result.maxLoc.x % matrix_output_columns)) / 
        matrix_output_columns
    );

    double y = (

        (image.rows() * 
        (result.maxLoc.x / matrix_output_rows)) / 
        matrix_output_rows
    );

    if( result.maxVal > 0.1d ) points.Add( new Point( x, y ) );
    else points.Add( null );
}

}

Line "Core.MinMaxLocResult result = Core.minMaxLoc( heat_map );" and next processing gets us only single person's pose. And comment for this line says the same. But how can I get the all local maximumns and detect all poses?

With best regards and best wishes, Sergey Netyagin

edit retag flag offensive close merge delete

Comments

does this (java?) code even work for a single person ?

i'm having some doubts, if output_image.get( i, 0, data ); is retrieving the right thing, imho, there should be a output_image=output_image.reshape(1,19); before trying to access it's data

berak gravatar imageberak ( 2018-04-09 12:06:32 -0500 )edit
  • you'll HAVE TO read the paper
  • it needs the original, linevec.prototxt,not opencv's shortened one (it does not have access to the PAF's)
  • i made an attempt at it lately, but there are still bugs
berak gravatar imageberak ( 2018-04-10 06:20:07 -0500 )edit
1

Yes..!! It works perfectly fine for a single person.

The code is available at this link: https://github.com/opencv/opencv/blob...

The step to reproduce is as follows: python3 openpose.py --input=COCO_val2014_000000000589.jpg --proto=openpose_pose_coco.prototxt --model=pose_iter_440000.caffemodel --dataset=COCO

The data is available at:

--> COCO_val2014_000000000589.jpg

https://github.com/CMU-Perceptual-Com...

--> openpose_pose_coco.prototxt

https://github.com/opencv/opencv_extr...

--> pose_iter_440000.caffemodel

http://posefs1.perception.cs.cmu.edu/...

Gajjar_Mihir gravatar imageGajjar_Mihir ( 2018-04-11 06:16:08 -0500 )edit

i know, i know.

i was only questioning your java version of it. (are you sure, you got the heatmap extraction right ?)

berak gravatar imageberak ( 2018-04-11 06:43:47 -0500 )edit
2

I am trying to implement it on android (java). The code works on ubuntu with opencv 3.4.1 but I am getting error on android with opencv 3.4.1. I have explained the error here: https://github.com/opencv/opencv/issu...

I tried the code on some images till now. It does fail sometimes.

Any help would be appreciated.

Gajjar_Mihir gravatar imageGajjar_Mihir ( 2018-04-11 06:54:51 -0500 )edit

could you add that part of the code to your question ?

(btw, your github repo is no more available)

berak gravatar imageberak ( 2018-04-11 06:58:36 -0500 )edit

Should I upload the Android project on my Github repo or should I paste the code here?

I think uploading would be a better option because the entire code would be visible there with indentation.

Gajjar_Mihir gravatar imageGajjar_Mihir ( 2018-04-11 08:31:49 -0500 )edit

a few lines with the result parsing here AND your whole code on github, maybe ?

berak gravatar imageberak ( 2018-04-11 08:58:44 -0500 )edit
1


int frameWidth = mat1.width();
int frameHeight = mat1.height();
Log.d("DEBUG",frameHeight + " " + frameWidth + " " + mat1);
Mat inp = Dnn.blobFromImage(mat1, 1, new Size(368, 368), new Scalar(0,0,0),false,false);
Log.d("See Hererere", inp.height() + " " + inp.width() + " " + inp.channels());
net.setInput(inp);
Mat out = net.forward();
Log.d("OUTPUT", out + "");

I am getting error in net.forward()

I have uploaded the entire code on github.

Gajjar_Mihir gravatar imageGajjar_Mihir ( 2018-04-19 05:54:50 -0500 )edit

github link ?

maybe you can try this one in the meantime

berak gravatar imageberak ( 2018-04-19 06:07:55 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
1

answered 2018-07-16 15:02:48 -0500

Roma gravatar image

OpenPose example in python working with multi-person (3 files) : https://gist.github.com/buff4life123/...

run "openpose_multiple_person"

@berak @Gajjar_Mihir @Sergey Netyagin

edit flag offensive delete link more

Comments

the scripts may have some not used lines of code (so you can remove that lines)

Roma gravatar imageRoma ( 2018-07-16 15:08:21 -0500 )edit

@Roma, cool !

berak gravatar imageberak ( 2018-07-17 05:04:45 -0500 )edit
Login/Signup to Answer

Question Tools

3 followers

Stats

Asked: 2018-04-09 09:29:15 -0500

Seen: 922 times

Last updated: Jul 16 '18