# Tracking of object not accurate enough

In short, I am trying to track a fish in a 30min video (see screenshot below for an example). I set-up everything to tracking should be as easy as possible. I recorded a video with python with settings to create an almost blank video with only the fish having a dark shape.

I have written a script in python using opencv to track the fish using various methods including background subtraction, blurring, dilation, and tresholding. However, even with the very low bg noise videos I still can't track the fish 100% of the time without also tracking noise. Without background subtraction tracking of the fish is 100% but it also tracks some darker areas in the tank, and with background subtraction it somehow keeps on having trouble tracking the fish all the time but now doesn't track the background noise.

I hope somebody with more experience on this can help me with this. I thought with the clear video it shouldn't be too hard to do. A section of the 30min video can be found at http://mudfooted.com/testvid.mov if you'd like to try and this is my latest script:

import os
import cv2
import sys
import glob
import numpy as np
import csv
import shutil
from time import gmtime, strftime

#set standards
MINAREA = 250
MAXAREA = 5000

#set background image subtractor
bgs = cv2.BackgroundSubtractorMOG()

#Get list of video files
files = glob.glob("*.mov")

kernel = np.ones((8,8),np.uint8)
for f in files:

#Set-up writing of tracked video
fourcc = cv2.cv.CV_FOURCC('m', 'p', '4', 'v')
video = cv2.VideoWriter()
videoname = "".join([f[:-4],"_tracked.avi"])
video.open(videoname, fourcc, 12, (1420,690), True)

cap = cv2.VideoCapture(f)
tracklist = []
trajnr = 0
lastframe = 0

#Go through list of video frames
while True:
if flag == 0: #Something is wrong or end of the video file was reached
break

#Set the current timeframe
currframe = round(cap.get(cv2.cv.CV_CAP_PROP_POS_FRAMES)/12,2)

#Stop tracking after 30min
print currframe
if currframe > 1800:
break

# Modify image for tracking
cropped = frame[0:690, 0:1420] #crop image
img = cv2.GaussianBlur(cropped,(15,15),0) #blur image

#show image-bg image
fgmask = bgs.apply(img, None, 0.05) #low learning rate will account for fish not moving
img_tresh = cv2.threshold(fgmask, 0, 255, cv2.THRESH_BINARY)[1] #treshold image
img_dil = cv2.dilate(img_tresh,kernel,iterations = 1) #dilate objects above treshold

#Get list of objects
contours,hierarchy = cv2.findContours(img_dil,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
cv2.drawContours(cropped, contours, -1, (255,0,0) ,1)

#Size filter for objects
for i in range(len(contours)):
cnt = contours[i]
M   = cv2.moments(cnt)
area = cv2.contourArea(cnt)
if (area < MINAREA or area > MAXAREA):
continue
else:

#Set the new trajectory nr if time gap exists of 1sec
if trajnr == 0:
trajnr = 1
elif (currframe - lastframe)>1:
trajnr = trajnr+1
lastframe = currframe

#Get the coordinates of the object
cx  = int(M['m10']/M['m00'])
cy  = int(M['m01']/M['m00 ...
edit retag close merge delete

I'm no python user ..... but in your code not see Any hierarchy usage ..... normally hierarchy work very well if you use it as part of filter with contour area..

( 2015-01-18 11:00:47 -0500 )edit

Thanks for your comment. Can you be a bit more specific? I.e. what do you suggest I implement? Thanks.

( 2015-01-19 03:16:33 -0500 )edit

This looks like a solvable problem, but let me just tell you that in the world of computer vision, most of the times, there is no such thing as 100% success rate, that is just unrealistic even for the simplest problems. You'll see in literature that error rate is something that is measured and accepted within certain ranges depending on the algorithm. Remember that even human beings make mistakes when identifying an object, let alone a computer :) Cheers

( 2015-01-20 05:19:23 -0500 )edit

Sort by » oldest newest most voted

There are a lot of different ways to attack this problem (as you've found out!). As gfx suggested, you could look at the hierarchies, which give you some extra information about the contours. Running some type of feature descriptor might also help (I imagine that dark shaded area will have less features in them than the actual fish).

I don't see any Regions of Interest filtering going on there -- where you only look for the target near where the last one was. It really cuts down on false positives and processing time. If it's not found, then you can iteratively lower thresholds and expand the ROI until you find the object.

If you want to look at some more pre-baked solutions, CamShift and Meanshift might provide all you need, as well as running a feature description and then applying homography.

I've had good luck with the MILTracker in the OpenCV contrib repository. They have a tracker interface there are quite a few different algorithms. If you compile OpenCV with that added in, you have a nice consistent API where you can pretty simply plug and play various trackers as well as write your own that conform.....that's only if you need to get pretty fancy though.

more

Thanks for your answer. Your suggestions make sense. However, would you be able to help me further by providing example code of how you would use hierarchies, running some type of feature descriptor, using the ROI filtering etc. I am still pretty new to opencv and am struggling to find the right sources myself. Thanks!

( 2015-01-20 04:20:55 -0500 )edit

Still haven't received any constructive coding help unfortunately..

( 2015-01-25 09:15:54 -0500 )edit

Official site

GitHub

Wiki

Documentation