JAVA / ANDROID: Trying to detect buttons (in a car). Which is the way to go? I tried many ways already.

#android

asked 2013-08-22 06:32:58 -0600

Max Muster
61 ●1 ●2 ●10

hello there. im pretty new to opencv, only started using is a few weeks ago.

here is the situation:

i want to write an android application that takes a picture (or captures the frame of a video) of a button in (for example) a car. after the button is detected, the app is supposed to tell you what that button does.

the last part is not the problem, but in fact my problem is how i can detect and differentiate all the buttons in a car?

what i have tried so far:

1) using featuredetection: this worked alright, but it takes about 5 seconds for the image to be processed / detected. => this is too slow for me.

2) using template matching, as it was suggested to me: well, this worked well if the input image was exactly the same as the template image (just not cropped). but once the input image was not exactly the same image as the template, it did not work.

3) training my own haar classifier: i was trying to train my own classifier for the black cap of a pencil for the start. i took 40 positives and 635 negatives. out of the 40 positives i generated 3000 samples with the perlscript "createsamples.pl". training went good up to the 20st stage, but there were still too many "random" false detections. so i trained further, and the "random" false detections got less, but now it started to detect everything else i took a picture of aswell. example result: Result of the detection . here you can see that the cap was detected (which i wanted to), but also the coin was detected (which i sure didn't want to). my positives looked like this: Positive pictures

so my question is:

does any of you have a suggestion about what the correct aproach for my goal is? i am kind of stuck as i have tried all the above methods but none was satisfying...

thanks in advance, i apprechiate any help :)

edit retag flag offensive close merge delete

Comments

Can you give some examples of images of buttons?

GilLevi ( 2013-08-22 08:20:03 -0600 )edit

thanks for your reply. the buttons will be all type of buttons which you can find in a car, like the buttons around the radio, the buttons for the navigation, basically every button in the dashboard of a car. for example buttons like those: http://i.imgur.com/DqvAL7L.jpg , http://i.imgur.com/190dzqU.jpg , http://i.imgur.com/SKR7MWF.jpg . depending on which way of reaching my goal i have to choose, i will take pictures of each button and store them (probably in a database or something).

Max Muster ( 2013-08-22 09:07:44 -0600 )edit

add a comment

2 answers

Sort by » oldest newest most voted

answered 2013-08-22 16:09:43 -0600

Guanta
6736 ●6 ●25 ●79

I think your third approach is actually one of the best ways. Two steps for improvement:

Use much more negatives (typically also much more than positives) for the cascade-classifier.
Try LBP / HOG instead of HAAR. HOG is actually kinda the state of the art for object recognition (maybe not the OpenCV version though since it has several restrictions in window size etc).

Finally, you could try try to train your own LatentSVM model (you'll find hints in this Q&A forum if you search for it).

edit flag offensive delete link

Comments

It's a bit problematic to train a new LatenSVM model I think. New models are trained in Matlab, and I haven't found any script to translate them to the OpenCV's xml's (the script in opencv extra is outdated and does not fit to the new LatentSVM matlab package).

Do you know of any new script to translate the models?

Thanks!

GilLevi ( 2013-08-22 16:16:06 -0600 )edit

@GilLevi: No sorry, haven't used it so far

Guanta ( 2013-08-22 16:32:53 -0600 )edit

@Guanta thanks for your reply. i have been using LBP instead of HAAR as featureType for my training. nevertheless, my training finished in about 2 hours, for 30 stages, 2000 positive samples and 635 negative samples. isn't that way too fast? i heard it should take many more hours, or even days (atleast with the old deprecated training method). is there something wrong if it is that fast with my current parameters?

Max Muster ( 2013-08-23 01:13:39 -0600 )edit

Imho you have too few negatives for 30 stages so I guess no information is added in the last stages. @StevenPuttemans: Do you have another idea?

Guanta ( 2013-08-23 03:10:50 -0600 )edit

well, the training of the last stages (25 - 30) still took one of the two hours. basically, i just dont know how to avoid such false detections like that: http://i.imgur.com/ywxYWNL.jpg .it would be awesome if you guys know anything i could do better to what i did so far. meanwhile i will try with more negatives, like 4000 should be enough?

Max Muster ( 2013-08-23 03:56:31 -0600 )edit

It is simple, his negatives and positives are disciminative enough to find different features. However, if a large set like that trains that fast, it will give alot of false positives I am guessing, since the negative space of your SVM will be proportioned way to small. Also, don't use rubbish negatives. I have seen people feeding images of animals as negatives when they are detecting fruit in a wearhouse. You should think about your application and only use meaningfull negative images. Like in your case, multiple inside views of a car with the buttons erased, street views from a car while driving, ...

StevenPuttemans ( 2013-08-23 04:08:38 -0600 )edit

@StevenPuttemans thanks for your answer. i understand that i have to use meaningfull negatives. so you say, that if i use better negative images, it will help to solve my problem? i didnt completely understand the first part of you comment, sorry for that. also, how many positives and negatives do you suggest for each button in this case?

Max Muster ( 2013-08-23 07:09:39 -0600 )edit

It depends on the actual amount of variance in the images and the objects. This is actual my active phd research topic, and I did not yet found a perfect answer. However, it seems that object and scene variance are the main drives of choosing correct samples as positives and negatives.

StevenPuttemans ( 2013-08-23 09:11:50 -0600 )edit

@StevenPuttemans yeah it seems so. if you get the perfect positives and negatives, then the training will be very effective and the detection / recognition will be working good. so what you say, is that at the moment we can't say how many positives / negatives we need and how exactly they have to look, right? thanks again.

Max Muster ( 2013-08-23 11:53:43 -0600 )edit

I have some ideas and research results :) but first get them plublicated and then I will be happy to share them. However, stating exactly how many and also which samples you need, is nearly impossible for the moment.

StevenPuttemans ( 2013-08-24 15:51:40 -0600 )edit

see more comments

answered 2013-08-22 09:15:54 -0600

GilLevi

1357 ●10 ●27 http://gilscvblog.com/

Not an easy problem and it's not clear (at least, to me) what is the best approach.

However, having seen some example images, I can suggest detecting shapes in the image according to the shapes of the buttons. Look for shapes in the given image that are of the size and shape you expect.

See here: http://opencv-code.com/tutorials/detecting-simple-shapes-in-an-image/

edit flag offensive delete link

Comments

thanks for your answer. the aproach with the shapes sounds good to me, nevertheless the size will depend on how far or close the user gets when he takes the picture of the button, right? i guess i can't know that :/ also, many buttons have the same shape, many of them are like circles, and many others are square, so it would be hard to differentiate buttons with the same shape aswell i guess. thanks again for your reply, i will be grateful for every further help :)

Max Muster ( 2013-08-22 13:48:10 -0600 )edit

I believe the code in the link detect's shapes at all scales. But you take a look to make sure.

regarding differentiating between different buttons with the same shape - you asked how to detect the buttons and for that the shape approach will do. You also need to recognize them? I don't understand.

GilLevi ( 2013-08-22 13:55:13 -0600 )edit

thanks again for your reply. yes i also want to recognize them, like i stated in my first post. the app is supposed to tell me, what exactly each button is for :)

Max Muster ( 2013-08-23 01:10:16 -0600 )edit

Well, perhaps you can recognize them according to the icon printed on them. For that, you'll need to train a classifier. I'm not that familiar with classification with OpenCV, so maybe someone else can answer.

GilLevi ( 2013-08-23 05:55:53 -0600 )edit

add a comment

JAVA / ANDROID: Trying to detect buttons (in a car). Which is the way to go? I tried many ways already.

Comments

2 answers

Comments

Comments

Links

Question Tools

Stats

Related questions

JAVA / ANDROID: Trying to detect buttons (in a car). Which is the way to go? I tried many ways already. edit

Comments

2 answers

Comments

Comments

Links

Question Tools

Stats

Related questions

JAVA / ANDROID: Trying to detect buttons (in a car). Which is the way to go? I tried many ways already.