panc's profile - activity

Hi guys,

I'm stuck again on my app, i.e. detecting boxes on shelf.

now I am starting to use simpler photos, with foreground (boxes) is different from background: colored boxes on white or black background.

Using canny and findContours did not give me correct result even with only 3 boxes on the shelf, so now I am thinking to detect all the lines in the img using hough, and then filter them by:

1) lines can be horizontal and vertical only (near 90 degrees and near 180 degrees) 2) calculate intersection with lines 3) detect if there are some rectangles in the resulting image

Now the question, how can I do the three steps above?

Thank you so much, this is my last exam at the university and on july i'll graduate (hopefully), so I am a little bit anxious with the delivery of this last project for my last exam!!!

my problem is that I have to detect all the products (rectangles and possibly bottles, but mainly rectangles) on a supermarket shelf. I've tried it all the possible solutions!

Hi guys,

I'm working with OpenCV4Android but i'm experiencing so problems on correctly understand ORB methods and what can I know from that class.

Most of all, I want to know if: passing a generic shape as template, ORB can detect elements that have the same or similar shape, regardless the foreground (i.e. can I get a metric from ORB that gives to me something like "this element has 5/15 matches", so that I can give a correct threshold for my algorithm?)

I've already tried to use classical object recognition to detect shapes using hough, canny, harris, etc etc but they give me only s*ty results.

Thank you guys!

thank you so far. I didn't know that is not so simple as I thought. In your opinion which type of methodology can i follow to achieve my goal?

hi, thank you for the reply.

I know that sift and surf are nonfree, but I didn't heard that they arent implemented in java, i know that I have to recompile openCV with the nonfree module.

In your opinion, which are the best methodologies for this type of problem? thanks!

Hi guys,

I'm working again on a shelf detector (detects product on a shelf and cut them into images) but the last approach (using hist inspection, as mentioned in my previous question) did not suited for my purposes.

So, my tutor told me to try to use an object detection algorithm like SIFT or SURF and use the features extracted from a template image to detect other products.

Because the products differ in their "contents", i.e. logos and drawings on their covers, my idea is to use object detection and then tell to do opencv something like this:

1)get descriptors 2)match all the part of the test image that has a % of keys (not 100% match) > of TOT%

so I have to write an algorithm that selects objects in the test image that has a % of keys matching the template image.

Is this possible with opencv?

Thank you in advance, Panc

Oh sorry, thank's for the correction!

As you can see from your stacktrace, openCV cannot find an implementation of Rect if you use this prototype :

Imgproc.rectangle_1(long, double, double, double, double, double, double, double, double, int)

Looking at the official documentation here, openCV expects something like this:

Imgproc.rectangle(Mat src,Point pt1, Point pt2, Scalar color, int thickness, int lineType, int)

and so its prototype is Imgproc.rectangle(Mat,Point, Point, Scalar, int, int, int) that is different from yours.

Alternatively, passing a Rect instead of Points:

 Imgproc.rectangle(Mat src,Rectangle rect, Scalar color, int thickness, int lineType, int)

and so its prototype is Imgproc.rectangle(Mat,Rect, Scalar, int, int, int) .

I think that yu have to double check your code to match these prototypes.

Other approaches ? can you explain them? thank you so much!

Uhm, so do you think that if I do preprocessing like the link that you posted, and then use your approach will result in more image-independent solution?

thank you!!

Do you think that this approach works even with captured images?

EDIT: for types mismatch I've found a simple solution to pass from Mat to List<point> without the usage of MatOfPoint and .toList() method:

    Mat pts=new Mat();
    List<Point> pts_ref=new ArrayList<>();
    Converters.Mat_to_vector_Point(pts,pts_ref);

Now I'm tyring to modify canny's thresholds, using a median method to calculate the median value of the grayscale image and then give low_threshold=median0.66 and hig_threshold=median1.33 .

I think it is acceptable, because previously I was able to view the results of reduce , so i think REDUCE_AVG will work well.

UPDATE: after few logs, I've found that after canny, canny.cols() gives me count 1. strange

zero! it can be a type error, because you use vector<point> as 2nd parameter of findNonZero, but Java's impementation of findNonZero accepts a Mat (not properly a MatOfPoint, but i don't know how to convert a Mat in a List<point> in other methods).

I know that are zero rects because I print a log just after rois= divideHW( clone, true, 0, 255 );printing rois.size(), that gives me 0, so no rects inside.

UPDATE 1

Today I debugged a little and found out some problems that, probably, in c++ don't shows up.

inside this part of divideHW method:

for( int i=0; i< pts_ref.size(); i++ )
    {
        if( dim )
        {
            if(i!=pts_ref.size()-1 ){
            rect.height = (int)(pts_ref.get(i).y-ref_y);
                if(rect.height>100)
                {
                Rect r=rect.clone();
                rois.add(r);
                }
            rect.y = (int)pts_ref.get(i).y;
            ref_y = rect.y;
            }
            else
            {
                rect.height = gray.rows() -(int) pts_ref.get(i).y;
                  if(rect.height>100)
                {
                Rect r=rect.clone();
                rois.add(r);
                }
            }
        }

        else
        {
            if( i != pts_ref.size()-1 )
            {
                rect.width = (int) pts_ref.get(i).x - ref_x;
                if(rect.width>50){

                    Rect r=rect.clone();
                    rois.add(r);

                }
                rect.x = (int) pts_ref.get(i).x;
                ref_x = rect.x;
            }
            else
            {
                rect.width = gray.cols() - (int)pts_ref.get(i).x;
                if(rect.width>50){

                    Rect r=rect.clone();
                    rois.add(r);

                }
            }
        }

    }

I've added a clone() call to our just calculated rect, to avoid that pushing the rect without cloning, once the for loop finishes, all the N entries of rois are equal to the last rect analyzed..

Second, I've managed to control the minimum width and heights of rects, to avoid getting solutions not consistent (i.e. part of the background or part of unwanted boxes.

Now I succesfully can get my boxes, but not exactly. I can get a correct box, or multiple boxes together, or part of boxes (like half a box or a quarter of a box).

I will try now with another test image, then I will test if this approach works even with captured images from camera (taking photo in "perfect" front of a shelf, to avoid perspective as mush as we can).

test 2 image

Ok, i understood.

So I've already got the four "rows", and now for each row I have to canny and then findNonZero. I'll try it. Because of I'm working with Opencv4Android my work is a little bit slower due to translation to java process!

Oh, great! Can you briefy explain the methodology you have followed? not a code description :)

OK, I hope you will have better luck with Canny!

Ok, I've managed to get your same solution, and for now it gives me the same results as yours. Now my idea is to use a gradient to detect when theres a huge change in that values produced by reduce (so there's a line) and then plot it. Do you think it could be a good approach? I'm using Sobel derivatives on x and y axes, but for now I cannot compute its values (it returns me only zeroes).

Hi guys,

I'm quite new to Javacv, and I'm stuck on a really simpe problem:

I have a Mat, and I want to save it to my Android device. All other googled solutions don't work for me, so I can I convert a Mat element to an IplImage?

    Mat gray;
    gray=new Mat();

    cvtColor(originalMat, gray, CV_BGR2GRAY);
    //here I want to save my Mat as an image

Thank you in advance!

I know, I don't have to call "line" as you wrote, but I am porting this code to java and i'd like to know why you set p oints coordinates as done in the code. Is the only piece of code that I don't understand (semantically).

hi, can you explain this part of code?

line( reduced_h_graph,Point(i,0),Point(i,reduced_h.at<uchar>(0,i)),Scalar(255,255,0),1);

I understood that it draws a line, but i don't understand clearly when you write "reduced_h.at<uchar>(0,i).

sorry - I understood

Really really interesting...Iìll try this and i'll tell you if it solved my problem!

Yes, i understood, but i think that, assuming that i take an image quite perfectly in front of a shelf, they can be viewed as boxes, because there is not a perspective on the circled shape! :)

Sorry, I'm italian, and for my english cans are like boxes! ahah, yeah , i'll post a possible output.

Hi all, i'll repost here a question posted on StackOverflow (without real responses), I'll hope i'll be luckier here!

I'm developing an Android app that recognizes, from a store shelf image, all the boxes (products) present on the shelf.

My approach so far is the following:

1. grayscale
2. bilateral filter (or GaussianBlur, but I've found that using bilateral filter is better to preserve edges)
3. adaptive threshold
4. dilate (don't know if it's necessary)
5. canny
6. findContours

So, if the source image is simple, (like a b/w drawing of a shelf and some boxes) it can detect them, but with real images of shelves it isn't working.

The main problem is that single boxes have different "foreground" colors and logos, and my steps detect also all the edges of the "inner" box (i.e., the colours inside the edges of the box) and gives me totally nosense results. For simplicity I'll show belows my intermediate results and the source image:

1) source

2) grayscale

3) filtered (gaussianBlur in this case)

4) adaptive threshold

5) dilation

6) canny

As you can see, because I cannot remove the foreground of each box, also all the edges given by the logos or text come into play and noises my results.

How can I overcome this problem?

My ony idea is trying to "remove" or decolouring the inner boxes, but i don't konw how to do it! Thanks to all!

P.S. please don't reply with links relative to the already tried tutorials found on this website. They didn't helped me solving the probem. Thanks!

EDIT:

A possible solution to that specific image could be (is really raw in this example cause i've done it manually ;) ):

And this app has to detect boxes and cans possibly, because ithout perspective (i.e. assume that I have images in front of the shelf), cans are viewed as boxes.

Hi all, i'm new about OpenCV but i have some experience in android developing. I'm a Politecnico di Milano Computer Engineer student and I have to graduate soon this year but i have to deploy an image analysis application in which i take a picture from a shelf and then detect all the products in the shelf.

Because our teacher haven't given us a correct documentation (he gave us nothing, only mathematical formulas) I'm here to ask you some tips or documentation that can help me in doing this homework.

thank you all in advance, and sorry if this question sounds a little strange.