Goosebumps's profile - activity

I have a robot holding a chessboard in front of a camera. In know the chessboard's position in the robot world. If I do pose detection of this chessboard I also now it's position in the camera world. Both cooridnates together define the relation between camera world and robot world. This all works okay, but of course, there are some tolerances.

My question is related to the detection of the chessboard pose with the camera. What is the best way to position the chessboard in view of the camera:

1. Perpendicular
2. Rotated around 1,2 or three axes of the camera world
3. Other.

Which will give the most reliable results?

Have a look at watershed segmentation in combination with distance transform. There are many examples on the interwebz. Here is one in OpenCV and python: watershed

Most common steps: - Use automatic global thresholding such as Otsu - Erode this, distance transform and threshold it to get sure foreground - Dilate it to get sure background - Do watershed segmentation using the sure fore- and background

Because it is possibly easier than detecting the specific vehicle itself. This is not the same as detecting a bus. He wants to detect three specific busses, I presume, not the busses that happen to look like those three. The one thing that identifies a bus is its licence plate. Think about it, it is really not such a strange suggestion.

How about this: 1. Find rectangles that have an aspect ratio roughly that of a licence plate. 2. Do OCR on the licence plate 3. Decide of the licence plate is one of those 3 busses

PS Flashing with IR helps a great deal with licence plates if you remove/leave out the IR filter of your camera

It reminds me of this tutorial on watershed: marker-controlled-watershed-segmentation It uses morphological reconstruction though, which is not standard available in OpenCV. The following thread may be useful for this: morphological-reconstruction

Yes, but it is ever so much fun ;) BTW this is Emgu

If found that the morphological reconstructions are not that hard to implement if you read the following document: Morphological Image Processing

This one is also nice because it has some example images of what to expect Morphology

The clue is that the reconstructions' main ingredients are geodesic erosion and dilation. These are basically masked erosion and dilation by using a min operator. Code snippet:

Image<Gray, byte> m = image.Dilate(size, shape); // normal dilation
Image<Gray, byte> geoDilate = m.Min(mask); // masking operation

Same for erosion but using a max operator. Now dilation by reconstruction is keep on geodesically dilating until the image doesn't change anymore.

Image<Gray, byte> m0;
Image<Gray, byte> m1 = image;
do
{
    m0 = m1.Clone();
    m1 = m0.GeoDilate(mask, size, shape);
} 
while (!m1.Equals(m0));
Image<Gray, byte> DilateRec = m1;

Finally, opening by reconstruction is first erode the image. The eroded image is then dilated with the original image as mask:

Image<Gray, byte> m = image.Erode(size, shape);
Image<Gray, byte> openRec = m.DilateRec(image, size, shape);

Closing by reconstruction is the other way around.

Nice links. This library seems to be capable of doing the reconstruction. However, I was hoping to stay within Emgu (C# opencv).

Yes, that is the operation I was looking for.

Does anybody know how to do morphological reconstruction using OpenCV (or emgu). Of course I can find the algorithm and implement it from scratch, but I was hoping this is not neccessary.

Some background. In matlab there is the imreconstruct operation. See: matlab imreconstruct example

It can do the following. The source image:

Eroding it gives

Reconstruction yields those characters that left something after erosion

Credits for the images go to Steve Eddins (as in the link above)

Try watershed: watershed example

Would be nice if you would post some feedback on your progress

You state that you do not want to use morphological operations. I am assuming this is to keep the original shape identical. If so, you could still consider using opening by reconstruction. Have a look over here: opening-by-reconstruction

I don't know how to do it in OpenCV though. Anyone?

A morphological operation that could work is opening:

        int morphOpSize = 3;
        StructuringElementEx element =
            new StructuringElementEx(
                morphOpSize,
                morphOpSize,
                morphOpSize / 2,
                morphOpSize / 2,
                Emgu.CV.CvEnum.CV_ELEMENT_SHAPE.CV_SHAPE_ELLIPSE);

        filtered = gray.MorphologyEx(
            element,
            Emgu.CV.CvEnum.CV_MORPH_OP.CV_MOP_OPEN,
            2);

(sorry for my Emgu dialect)

Otherwise look for edge preserving low pass filtering.

If you pick up a lot of noise, maybe you could have a look at blob tracking augmented with kalman filtering.

Any luck yet? I'm curious how you solve this.

You may want to start using a top hat operation. That will make you get rid of the blurry non black background.

Second step would be binarization. I used a simple threshold, but there is still some work to be done here. Perhaps watershed could help you.

Then you may want to use findContours and do away with blobs that are more then e.g. 1.5 times as high as they are wide.

static void Main(string[] args)
{
    Image<Bgr, byte> bgr = new Image<Bgr, byte>(@"k:\tomgoo\im.png");
    Image<Gray, byte> gray = bgr.Convert<Gray, byte>();

    // Perform top-hat
    int morphOpSize = 5;
    StructuringElementEx element =
        new StructuringElementEx(
            morphOpSize,
            morphOpSize,
            morphOpSize / 2,
            morphOpSize / 2,
            Emgu.CV.CvEnum.CV_ELEMENT_SHAPE.CV_SHAPE_ELLIPSE);

    Image<Gray, byte> filtered;
    filtered = gray.MorphologyEx(
        element,
        Emgu.CV.CvEnum.CV_MORPH_OP.CV_MOP_TOPHAT,
        1);

    // binarize image
    double thresh = CvInvoke.cvThreshold(
                    filtered,
                    filtered,
                    100,
                    255,
                    THRESH.CV_THRESH_BINARY);

    // find and filter out blobs
    Contour<Point> contours = filtered.FindContours(
        CHAIN_APPROX_METHOD.CV_CHAIN_APPROX_SIMPLE,
        RETR_TYPE.CV_RETR_CCOMP);

    Contour<Point> cont = contours;
    while (cont != null)
    {
        // Remember the next contour for later and isolate the current one
        Contour<Point> next = cont.HNext;
        cont.HNext = null;

        MCvBox2D minAreaRect = cont.GetMinAreaRect();
        if (minAreaRect.size.Height < 1.5 * minAreaRect.size.Width)
        {
            // Generate random color
            CvInvoke.cvDrawContours(
                bgr,
                cont,
                new MCvScalar(255, 0, 0),
                new MCvScalar(255, 0, 0),
                2,
                2,
                LINE_TYPE.EIGHT_CONNECTED,
                new Point(0, 0));
        }

        // Now go to the next contour
        cont = next;
    }
    bgr.Save(@"c:\out.png");
}

(Sorry for my emgu dialect.) You will get this result: and

Any luck so far?

I don't know if it will do you any good. But I think what you are looking for is related to the Gestalt principle of Good Continuation. See http://en.wikipedia.org/wiki/Principles_of_grouping#Good_Continuation

Looks like convolution with a square kernel or 2D moving average filtering if you want some google keywords. This is a low pass filtering technique. Since the kernel is square it will act differently depending on the orientation of your scratch. You might also consider a circular or gaussian kernel if this is your way to go (personally, I do not think so).

You could use what I call an oriented high-pass filter (don't know the official name). I pasted the code below. It is in Emgu though (.Net wrapper for OpenCV). The basic idea is that you take a high pass filter kernel like:

k = [-1 -1 4 -1 -1
     -1 -1 4 -1 -1
     -1 -1 4 -1 -1
     -1 -1 4 -1 -1
     -1 -1 4 -1 -1
     -1 -1 4 -1 -1
     -1 -1 4 -1 -1
     -1 -1 4 -1 -1]

and convolve it with the image. Then rotate the kernel by N degrees (e.g. 15) and convolve it again. You do this up to 180 degrees and take the max of all the convoluted images. Scratches will have a high response to this kernel when aligned with the rotated filter. Your grains will too though. But you could experiment with leaving out the direction of the grains... BTW, smoothing the image beforehand gave me better results.

private Image<Gray, float> OrientedHighPass(Image<Gray, byte> image)
{
Image<Gray, float> filtered = new Image<Gray, float>(image.Size);
Image<Gray, float> final = new Image<Gray, float>(image.Size);

// import parameters
int smoothOpSize = 5;
int filterLength = 16;
int angleStep = 15;

// construct the kernel
float[,,] k3 = new float[1, 5, 1]
{
    { { -1 }, { -1 }, { 4 }, { -1 }, { -1 } },
};
Image<Gray, float> kRow = new Image<Gray, float>(k3);
Image<Gray, float> kImageOriginal = kRow;
for (int l = 0; l < filterLength - 1; ++l)
    kImageOriginal = kImageOriginal.ConcateVertical(kRow);

// first step, smooth image
image._SmoothGaussian(smoothOpSize);

for (int angle = 0; angle < 180; angle += angleStep)
{ 
    // Create convolution kernel
    Image<Gray, float> kImage = kImageOriginal.Rotate(angle, new Gray(0), false);
            // make sure the average of the kernel stays zero
    kImage = kImage - (kImage.GetSum().Intensity / (kImage.Width * kImage.Height));

    float[,] k = new float[kImage.Height, kImage.Width];
    Buffer.BlockCopy(kImage.Data, 0, k, 0, k.Length * sizeof(float));
    ConvolutionKernelF kernel = new ConvolutionKernelF(k);

    filtered = image.Convolution(kernel);

    final = filtered.Max(final);
}

return final;
}

If you want to read out the meter using computer vision you might also consider transforming the image to a polar plot using logpolar.

This might make measuring easier:

Or if it is just the dots, you might even draw them here and transform back to cartesian. Your dots will be slightly deformed though.