Best features to track fish underwater
Hello everyone,
I am currently working on a project that aims to detect fish and track them in a video. For example, draw a white box around a shark and red one around a Thuna. I have good quality pictures of the fish to track. And I'm trying to know which techniques may help me the best and provide robust tracking results on a video in acceptable response time (1 sec to detect is acceptable). There are some information that I think may be useful for you to help me :
- Colour should be taken in consideration because some fish species have same shape with different colors;
- Input video quality is quite good;
I would like to know which features would fit this application. I have thought for example to mix SURF and color image moments. Any ideas are welcome, if you can add examples of working similar application to your ideas, I would greatly appreciate it.
video frame :
image from database:
Thank you all.
![[1]](http://farm9.staticflickr.com/8440/7949039830_f1731e12f7_o.jpg){kind=link}
![[2]](http://farm9.staticflickr.com/8437/7949038816_d219e8d178_o.jpg){kind=link}
Post some frames. It's more than helpful.
Added 2 pictures to illustrate a video frame and a picture from the database. It possible to have several pictures of the same fish if necessary. Images have good resolution and no watermark. video is at least 640x480px 25fps .
In my opinion SURF and similar algorithms will not work well in this case because of distortions made by the water and glass and because the live fish appearance changes too much - the fish will likely be in very different positions relative to the camera and also its body is not rigid - when it swims it bends and features change. Something based on color will be good, I think. Have you considered machine learning based approaches?
Thank you for posting Rui Marques, which approaches are you thinking about ? You mean SVM ? haar classifiers ? Neural networks ? If you could give me technique names and algorithms names and examples, it can also help me to look and read about them to decide if they are adequate to what I'm doing.
Here is a description of two major detection frameworks http://answers.opencv.org/question/877/how-to-match-2-hog-for-object-detection/#882 I hope it will help you, but I think that finding the best approach is a good research topic in itself - it's not something you will receive here in a short answer, but something you'll discover after a few months of work :) trying more possibilities and learning on the way. Something that most probably will not work is contours-based and template matching.
What do you think about what is done here ?
http://www.youtube.com/watch?v=3XHzjT_4oIA&feature=context-chv
Very useful comment @sammy , I guess it's better to use texture descriptors in our case, which one you think might be better here ?
First, in the video they say they use KLT=lucas-kanade tracker. Second: between tracking and recognition is a huge complexity difference - the first one is an hour of work. Finally, I would say that texture descriptors is a good way, but it requires a lot of work. By example, to train a decent face detector you need at least a few hundreds images, well cropped and correctly aligned.
About that video, my guess is that it is tracking the eye of the fish by detecting the eye as a semi-circular blob. Meaning, no fish recognition is done, it is just following shapes that "kind of" look like eyes. About machine learning approaches, i didn't mean any particular one, i do not have experience using ML so i do not know which one would be better for your problem.
You mean that he tracks fish using KLT but he doesn't recognize them ? That's not very useful in our case indeed. It's quite interesting to have your opinion on that because I don't and will not have hundreds of pics of every fish. But I think differences between fish species are bigger than differences between human (same specie) faces. You think it will require how many pics approximatively for every specie ? I wanted also to ask about optical flow, do you think it can be of any use here ? for tracking maybe ?
Anyway, I really like your explanations and advices @sammy.