Ask Your Question
2

How to detect individual body parts in an image

asked 2014-06-29 14:24:11 -0600

Buck gravatar image

updated 2015-12-29 13:50:39 -0600

Background:

I would like to find a way to identify individual body part limbs in an image (ie such as Forearm or lower leg). The only problem being that I can't seem to find a reasonable feature detector or classifier to detect this in a rotation and scale invariant way (as is needed by objects such as forearms). One solution I have tried so far to no avail is HOG detection for forearm identification. I used a 32x32 window with a variety of different input parameters but was never able to to retrieve accurate detection in images.

Information:

Lets focus on forearms for this discussion. A forearm can have multiple orientations, the primary distinct features probably being its contour edges. It is possible to have images of forearms that are pointing in any direction in an image, thus the complexity. So far I have done some in depth research on using HOG descriptors to solve this problem, but I am finding that the variety of poses produced by forearms in my positives training set is producing very low detection scores in actual images. I suspect the issue is that the gradients produced by each positive image do not produce very consistent results when saved into the Histogram. I have reviewed many research papers on the topic trying to resolve or improve this, including the original from Dalal & Triggs [Link]: http://lear.inrialpes.fr/people/triggs/pubs/Dalal-cvpr05.pdf It also seems that the assumptions made for detecting whole humans do not necessary apply to detecting individual features (particularly the assumption that all humans are standing up seems to suggest HOG is not a good route for rotation invariant detection like that of forearms).

Note:

If possible, I would like to steer clear of any non-free solutions such as those pertaining to Sift, Surf, or Haar. I have also asked the same question on Stack Overflow: http://stackoverflow.com/questions/16752709/how-can-hog-be-used-to-detect-individual-body-parts

Question:

What is a good solution to detecting rotation and scale invariant objects such as human limbs in an image? Particularly for this example, what would be a good solution to detecting all orientations of forearms in an image?

edit retag flag offensive close merge delete

Comments

2

Are you familiar with detection by parts A.K.A latent SVM? it has a model for detecting human's by part. Perhaps from the different filters response for body parts you can deduce the location of the corresponding body part.

GilLevi gravatar imageGilLevi ( 2014-06-30 08:32:26 -0600 )edit

1 answer

Sort by ยป oldest newest most voted
6

answered 2014-06-30 09:30:08 -0600

Like @GilLevi already pointed out, you are looking to the wrong set of algorithms. You need a body part detection algorithm designed for these purposes. Techniques that come in mind to me are

  1. LatentSVM, discovered by Felzenszwalb : http://www.cs.berkeley.edu/~rbg/latent/
  2. Class specific hough forests by Gall and Lempitsky : http://www.vision.ee.ethz.ch/~gallju/projects/houghforest/
  3. The buffy stickman approach of the Calvin group : http://www.robots.ox.ac.uk/~vgg/data/stickmen/
  4. The integral feature channel approach of dollar is also in possibility of detecting more complex models and could be combined with checking multiple rotations : http://vision.ucsd.edu/sites/default/files/dollarBMVC09ChnFtrs_0.pdf

But keep in mind that these techniques are active research topics, quite complex and they will not be available off the shelf in OpenCV. There is a basic LatentSVM detector class but it works rather slow and is totally not optimized (was based on the first implementation of Felzenszwalb)

edit flag offensive delete link more

Comments

Another area I have recently been researching is using a Feature Detection & Description though the likes of KAZE, BRISK, &/or FREAK (among other options). Since the areas you mentioned above aren't directly available in OpenCV, will utilizing say Feature Detection/Description + BOW be a viable alternative?

Buck gravatar imageBuck ( 2014-07-01 23:03:24 -0600 )edit

I don't think so, the current state of OpenCV isn't made for such complex computations. You can use it for parts of the approaches, but the overall still needs to be develloped manually. You could however look around the net for available C++ implementations. As to your suggestion, I think feature detection and description would work if you would have a way of localizing possible limb regions. If not, the matching of all possible situations will take ages and will be a very computational expensive algorithm.

StevenPuttemans gravatar imageStevenPuttemans ( 2014-07-03 02:14:17 -0600 )edit

+1, I would like to do an amelioration for one of my project based on this! What do you think about the new latent SVM from 2.4.11/3.0.0, is it more optimised?

thdrksdfthmn gravatar imagethdrksdfthmn ( 2015-06-11 05:08:01 -0600 )edit
1

I did not try it yet. It seems that the big difference between the available implementation of Felzenswalb and the OpenCV implementation is that OpenCV moved to the star based approach, but it is still not that optimized as the latent version. 2.4.11 still contains the old version afaik.

StevenPuttemans gravatar imageStevenPuttemans ( 2015-06-11 07:13:18 -0600 )edit

Question Tools

1 follower

Stats

Asked: 2014-06-29 14:24:11 -0600

Seen: 16,395 times

Last updated: Jun 30 '14