elmiguelao's profile - activity

There's also some code for Automatic Number Plate Recognition (ANPR) here in EmGu but very easily portable to OpenCV using contours. I think is oriented towards close range pictures, like those taken in public parking lots and then identifying the characters, which might or might not be what you are looking for.

what's the question exactly?

Grabcut output is the mask, not a complete segmented multichannel image.

What you need to do is convert the output mask into the alpha channel of your image; for this, best use "Split" on the input RGBA to separate the channels, discard the input A channel, and "Merge" back into the output RGBA the RGB channels from the input with the mask from Grabcut.

Or, if you want the mask applied to your input image, "Split" the channels like before, then Multiply each input RGB channel with the mask, and "Merge" then back like before.

As a side note, openCV GrabCut implementation for square foreground seeds works best if the rectangle is larger than the object to segment. Otherwise you might see an empty output mask...

You could try matching a shape you know against the global image (link to opencv documentation on matchShapes). It works solidly and fast and will work with scaling but not with rotations or perspective transformations.

This q&a can give you pointers to more sophisticated comparisons.

You've got to be careful with the lightning conditions, haar classifiers are sensitive to this. Besides, there are two parameters, the pyramid iteration scaling (think 1.1 in the example you mention), and, more importantly, the minimum detected size (cvRect(30,30) in the example). The minimum size should ideally be different from eyes and face, and even related to each other. Try just reducing it substantially and see if you get better results. Finally, I recall a while ago figuring out that the Haar classifier does not take into account ROI, keep that in mind if you want to use the classification in real time apps, i.e. you'd need to implement that part with a bit of IplImage manipulation.

There is this openCV page on face recognition via PCA. Preprocessing the input image for lightning equalisation and/or scaling might be trickier, you can try histogram equalisation, Y-channel (luma) only histogram equalisation, and for the scaling you can start with the face detection bounding box and use it as a reference.

I've tried using the thresholds (see this Q&A) skin_out = (H < 20) ^ (S > 48) ^ (V > 80), they should work better than those you are using. Of course with the dilate(5x5 window)-erode(5x5 window) afterwards, to join smaller patches. But the image you are showing is overexposed, so the Hue threshold is most likely failing in some parts, since the implementation in openCV yields a range [0..180}, you also will need to extend the Hue threshold to cover something like [170..180} ^ [0..20] or so.

Another alternative is to use Haar classifier for pedestrian detection, assuming your "people" are either standing or moving, see this link that includes some discussion about it. I have tried the classifiers myself and movement + Haar-detection + Kalman tracking works relatively ok as long as:

1. there are not too many pedestrians on the scene, and they don't oclude each other,
2. the pedestrians don't look too small in the image (camera too far)
3. the background is not too cluttered with objects looking like pedestrians :)

I have tried the "Head and shoulders" from this web site , it's different from those under the openCV codebase, but they are compatible. Even when setting the minimum size of detected "object" larger, it is slower than face detection (like 3x slower), but the detection is real good.

I think the problem can be formulated as a clustering of data into 2 categories (stressed, not stressed) (see answer to question in this q&a entry) -- main difference with that situation is that here we know how many clusters to create. The openCV method for clustering in N-dimension is cvKMeans2. The method is quite easy to use, just being careful to create a matrix where each row contains a sample vector, in this case the 9 points mentioned.

The results would be optimal if the underlying distributions are concentrated, i.e. there are indeed two states to distinguish. Computationally speaking k-Means is fast because you identify a termination criteria via number of iterations and/or solution accuracy; the larger the gradient the faster it would converge, and the gradient is basically the concentration of samples. LDA or similar approaches rely on creating a big matrix description of the system, then inverting the matrix or solving the large linear system at once, vs. the iterative approach of the k-Means algorithm.

If the image looks very grey, you could also try applying the equalizeHist on YUV image - Y channel, note that this will change the colours of the image but it might make it visually/computationally better (in some sense). A code snippet to try quickly (all _ptr are pointers to IplImage):

cvCvtColor( RGB_image_ptr, YUV_image_ptr, CV_RGB2YUV );
cvSplit( YUV_image_ptr, Ychannel_image_ptr, Uchannel_image_ptr, Vchannel_image_ptr, NULL);

cvEqualizeHist(Ychannel_image_ptr, Ychannel_image_ptr);

cvMerge( Ychannel_image_ptr, Uchannel_image_ptr, Vchannel_image_ptr, NULL, YUV_image_ptr);
cvCvtColor( YUV_image_ptr, RGB_image_ptr, CV_YUV2RGB );

3D value per pixel could be drawn as RGB colour, depending on the range of values you have. Would this representation be meaningful?

best in what sense? Fastest, most accurate...?

Before asking if openCV is available, bigger issue is that MSP430 appears to have no Linux porting. It makes sense since this microcontroller has no MMU -- uCLinux could be an option but the MSP has really little amount of RAM, see this discussion. We could discuss about the necessary steps to port openCV but I'm afraid they would imply great effort to port or reuse libraries that openCV expects to find during compilation and runtime. Easiest option would be to get the openCV C code of the operations you want to use and migrate them individually into your project under TI CCS or gcc/ Eclipse.

I have always had to do it myself in C:

void iplimage_from_cvmat(CvMat* input, IplImage * output)
{
  int x,y;
  for( x=0; x < output->width; x++ ){
    for( y=0; y < output->height; y++) {
      // note: CvMat is indexed (row, column) but IplImage is indexed (x,y)
      // so the indexes must be interchanged!
      cvSetReal2D( output, x, y, CV_MAT_ELEM(*input, uchar, y, x) );      
    }
  }
}

The resulting rho and theta are indeed one step away from the line(s) you are looking for in the image. They represent a line passing through the origin that is perpendicular to the line you want to find. This page has a great introduction to Hough transform concepts, and explains this point.

You might want to use cvHoughLines2 instead, which will allow finding segments instead of lines and a couple of other improvements.

In this example you find some code to draw the lines from rho and theta; the idea is simple: calculate a point of that line, namely x0 = rho cos(theta), y0 = rho sin(theta), and notice that the slope of the line is (-theta), cos(-theta)=cos(theta), sin(-theta)=-sin(theta), and the very large numbers are there to use integer arithmetic.

for( size_t i = 0; i < lines.size(); i++ )
{
    float rho = lines[i][0];
    float theta = lines[i][1];
    double a = cos(theta), b = sin(theta);
    double x0 = a*rho, y0 = b*rho;
    Point pt1(cvRound(x0 + 1000*(-b)),
              cvRound(y0 + 1000*(a)));
    Point pt2(cvRound(x0 - 1000*(-b)),
              cvRound(y0 - 1000*(a)));
    line( color_dst, pt1, pt2, Scalar(0,0,255), 3, 8 );
}

Thank you!

If you mean converting an image to a cartoon-like style (like in this page ), then it can be achieved via a posterize effect using the PyrMeanShiftFiltering method (http://opencv.willowgarage.com/documentation/miscellaneous_image_transformations.html#pyrmeanshiftfiltering) plus a smoothing operation.

From that page:

The function implements the filtering stage of meanshift segmentation, that is, the output of the function is the filtered “posterized” image with color gradients and fine-grain texture flattened.

True, now that I read that. I just meant that cvFindContours also will find loads of contours that a human eye would not find relevant. I will edit the answer.

You can use the Hough transform to detect straight lines in the frame. Once the main lines are found, and seems it would work pretty good according to your second picture, the angles can be postprocessed. Note that the Hough operation ("transform") will return loads of straight lines of different lenght in the picture, unwanted ones should be filtered out via line size.

There is this full tutorial in openCV with code included: http://docs.opencv.org/doc/tutorials/imgproc/imgtrans/hough_lines/hough_lines.html.

Zach, perhaps you could mark this question as solved? So other people can link to it as such :)

I cannot see the images, does it happen to anybody else?

Whenever the redistributable is installed, the msvcr90.dll is in c:\winnt\system32 (perhaps winnt is name something else). Worst case, you could just copy the .dll into the folder where the executable is :)

Also you can take a look at this article from Ms where it tells you a bit the story. You are not the first one to stumble upon the .dll mess, that for sure ;)

An alternative is to segment each number and letter, via contour extraction, and analyse their Hu moments against those you pre-calculate on each reference character. Hu moments have the advantages of being rotation, reflection and scaling invariant, hence the name Hu invariants. This method is computationally cheap (i.e. fast): moments are aggregated measures from the shapes (center of mass, center of inertia, etc) so comparisons are made number-to-number, as opposed to pixel-to-pixel comparisons which are more costly.

In this article the whole recognition process is explained step-by-step, with CV code. In the Bradski-Kaehler openCV book (Chapter 8) there is a discussion about Hu moments for character recognition. Finally there is a library function cvMatchShapes which would extract those moments and compare them internally.

Finally, if you search in internet for "Hu moments character recognition", there are many academical references, but they boil down to the explanations quoted above.

How many different situations do you want to distinguish? A priori I would think this is a N-dimensional clustering problem...

What you are trying to do is called Template Matching in the computer vision world. Check this tutorial (http://docs.opencv.org/2.4.2/doc/tutorials/imgproc/histograms/template_matching/template_matching.html) for some introduction and code to the topic.

A word of caution, I think I recall this method does not deal with template rotation or scaling, meaning that your template should be present with the same size somewhere in the big image to hit a match, not smaller nor larger. Besides, it obviously and definitely does not deal with projective transformations.

when I tried setImageROI for face detection, with this same idea, I noticed that the detect function ignores the ROI, back in openCV 2.3.1 or so. Just a word of caution :)

Using openCV, check this link: http://blog.damiles.com/2008/11/basic-ocr-in-opencv/, perhaps it would help.

Jitter is a term usually applied to transmission of video frames on a network. The image looks like a stereo vision of some kind on a mounted/moving platform where the image reflects vibrations. Can you provide more details on the source and definition of your problem?

If you know the amount of segments to extract, mean-shift segmentation via openCV k-means can do the trick, although the inclusion might be a problem to be dealt with separately. I made a post on k-means here http://answers.opencv.org/question/2628/can-opencvs-mean-shift-implementation-be-used-for/.

More info can be found in this slides: http://robots.stanford.edu/cs223b04/CS%20223-B%20L11%20Segmentation.ppt (slide 41 onwards).

Tentatively, there is the openCV method for finding k-means for an arbitrary set of points, but you would need to know how many clusters you want to find, and there could be convergence issues depending on how the data is spread and how the algorithm (iterative) starts. So it might be of no help to you ;)

I found this image condenses completely the tons of k-means mathematical theory :) (from http://www.cs.cmu.edu/~dpelleg/kmeans.html) :

First on the Haar output, I've always seen the face bounding box to include vertically from half the forehead to the chin approximately. Never the whole head; an experiment can be done by detecting a face partially sideways and downwards: the detected center is on the nose, i.e. not in the middle of the head, and the size gets reduced (largely, if very downwards) compared to the size of the head, with hair and all.

Another option is to use the facedetection radius, convert it to a bounding box and get the skin colour pixels inside and in an immediate neighbourhood, like, say, 1.25 times the width. Then use those skin pixels smartly :) like for instance using the camshift method on it.

For finding skin colour pixels, you can check in length the idea here (http://www.shervinemami.co.cc/blobs.html), but basically is an AND of the thresholded channels Hue, Sat and Value. From that link, I found that the rule works best slightly modified like:

skin_out = (hue < 20) ^ (sat > 48) ^ (val > 80), where   ^ mean pixels-wise AND

There are other rules working on RGB space but I have not seen great improvement compared to this one. The HSV rule has troubles with people looking very pale, but usually this can be corrected by a preprocessing stage of luminance-equalisation on the input image. This is recommendable anyway if you want to track real world sequences where camera autoexposure can cause non-linear colour changes that will confuse the Haar detection process, the skin detection, or both.

A sample function creating a skin/no-skin gray image, from the HSV version, would be (in C):

    
int compose_skin_matrix(IplImage* rgbin, IplImage* gray_out)
{
      // crappy static pointer, works, but best if done some other way.
      static IplImage* imageHSV = cvCreateImage( cvSize(rgbin->width, rgbin->height), IPL_DEPTH_8U, 3);
      cvCvtColor(rgbin, imageHSV, CV_RGB2HSV);

      static IplImage* planeH = cvCreateImage( cvGetSize(imageHSV), 8, 1);  // Hue component.
      static IplImage* planeS = cvCreateImage( cvGetSize(imageHSV), 8, 1);  // Saturation component.
      static IplImage* planeV = cvCreateImage( cvGetSize(imageHSV), 8, 1);  // Brightness component.
      cvCvtPixToPlane(imageHSV, planeH, planeS, planeV, 0); // Extract the 3 color components.

      // Detect which pixels in each of the H, S and V channels are probably skin pixels.
      // Assume that skin has a Hue between 0 to 18 (out of 180), and Saturation above 50, and Brightness above 80.
      cvThreshold(planeH , planeH , 20, UCHAR_MAX, CV_THRESH_BINARY_INV);     //(hue < 20)
      cvThreshold(planeS , planeS , 48, UCHAR_MAX, CV_THRESH_BINARY);         //(sat > 48)
      cvThreshold(planeV , planeV , 80, UCHAR_MAX, CV_THRESH_BINARY);         //(val > 80)

      // erode the HUE to get rid of noise.
      cvErode(planeH, planeH, NULL, 1);

      // Combine all 3 thresholded color components, so that an output pixel will only
      // be white (255) if the H, S and V pixels were also white.

      // gray_out = (hue < 20) ^ (sat > 48) ^ (val > 80), where   ^ mean pixels-wise AND
      cvAnd(planeH  , planeS , gray_out);   
      cvAnd(gray_out, planeV , gray_out);   

      return(1); // make it smarter ;)
}

msvcr90.dll was the Visual C runtime library for Visual Studio 2008. You should be able to get the library only by downloading the redistributable for free from here http://www.microsoft.com/en-us/download/details.aspx?id=29.

Hi Zach. You could use adaptive thresholding, which is now a "native" function in openCV (http://docs.opencv.org/modules/imgproc/doc/miscellaneous_transformations.html#adaptivethreshold),

The following link http://homepages.inf.ed.ac.uk/rbf/HIPR2/adpthrsh.htm has a detailed explanation on how to use the adaptation methods (mean, gaussian), but, basically, it boils down to using a threshold level that is computed in the vicinity of the pixel, as opposed to just a global one.

Perhaps some residual CUDA enabling in the openCV trunk, you can force to the cmake the option "-DWITH_CUDA=off" and see what happens.