Ask Your Question

Tracking of object not accurate enough

asked 2015-01-18 08:45:45 -0500

joljols gravatar image

In short, I am trying to track a fish in a 30min video (see screenshot below for an example). I set-up everything to tracking should be as easy as possible. I recorded a video with python with settings to create an almost blank video with only the fish having a dark shape.

image description

I have written a script in python using opencv to track the fish using various methods including background subtraction, blurring, dilation, and tresholding. However, even with the very low bg noise videos I still can't track the fish 100% of the time without also tracking noise. Without background subtraction tracking of the fish is 100% but it also tracks some darker areas in the tank, and with background subtraction it somehow keeps on having trouble tracking the fish all the time but now doesn't track the background noise.

I hope somebody with more experience on this can help me with this. I thought with the clear video it shouldn't be too hard to do. A section of the 30min video can be found at if you'd like to try and this is my latest script:

import os
import cv2
import sys
import glob
import numpy as np
import csv
import shutil
from time import gmtime, strftime

#set standards
MAXAREA = 5000

#set background image subtractor
bgs = cv2.BackgroundSubtractorMOG()

#Get list of video files
files = glob.glob("*.mov")

kernel = np.ones((8,8),np.uint8)
for f in files:

#Set-up writing of tracked video
fourcc ='m', 'p', '4', 'v')
video = cv2.VideoWriter()
videoname = "".join([f[:-4],"_tracked.avi"]), fourcc, 12, (1420,690), True)

#Start with tracking new video
cap = cv2.VideoCapture(f)
tracklist = []
trajnr = 0
lastframe = 0

#Go through list of video frames
while True: 
    flag, frame =
    if flag == 0: #Something is wrong or end of the video file was reached

    #Set the current timeframe
    currframe = round(cap.get(,2)

    #Stop tracking after 30min
    print currframe
    if currframe > 1800:

    # Modify image for tracking
    cropped = frame[0:690, 0:1420] #crop image
    img = cv2.GaussianBlur(cropped,(15,15),0) #blur image

    #show image-bg image
    fgmask = bgs.apply(img, None, 0.05) #low learning rate will account for fish not moving
    img_tresh = cv2.threshold(fgmask, 0, 255, cv2.THRESH_BINARY)[1] #treshold image
    img_dil = cv2.dilate(img_tresh,kernel,iterations = 1) #dilate objects above treshold

    #Get list of objects
    contours,hierarchy = cv2.findContours(img_dil,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
    cv2.drawContours(cropped, contours, -1, (255,0,0) ,1)

    #Size filter for objects
    for i in range(len(contours)):
        cnt = contours[i]
        M   = cv2.moments(cnt)
        area = cv2.contourArea(cnt)
        if (area < MINAREA or area > MAXAREA):

            #Set the new trajectory nr if time gap exists of 1sec
            if trajnr == 0:
                trajnr = 1
            elif (currframe - lastframe)>1:
                    trajnr = trajnr+1
            lastframe = currframe

            #Get the coordinates of the object
            cx  = int(M['m10']/M['m00'])
            cy  = int(M['m01']/M['m00 ...
edit retag flag offensive close merge delete


I'm no python user ..... but in your code not see Any hierarchy usage ..... normally hierarchy work very well if you use it as part of filter with contour area..

gfx gravatar imagegfx ( 2015-01-18 11:00:47 -0500 )edit

Thanks for your comment. Can you be a bit more specific? I.e. what do you suggest I implement? Thanks.

joljols gravatar imagejoljols ( 2015-01-19 03:16:33 -0500 )edit

This looks like a solvable problem, but let me just tell you that in the world of computer vision, most of the times, there is no such thing as 100% success rate, that is just unrealistic even for the simplest problems. You'll see in literature that error rate is something that is measured and accepted within certain ranges depending on the algorithm. Remember that even human beings make mistakes when identifying an object, let alone a computer :) Cheers

Pedro Batista gravatar imagePedro Batista ( 2015-01-20 05:19:23 -0500 )edit

1 answer

Sort by ยป oldest newest most voted

answered 2015-01-19 22:34:54 -0500

Petty gravatar image

There are a lot of different ways to attack this problem (as you've found out!). As gfx suggested, you could look at the hierarchies, which give you some extra information about the contours. Running some type of feature descriptor might also help (I imagine that dark shaded area will have less features in them than the actual fish).

I don't see any Regions of Interest filtering going on there -- where you only look for the target near where the last one was. It really cuts down on false positives and processing time. If it's not found, then you can iteratively lower thresholds and expand the ROI until you find the object.

If you want to look at some more pre-baked solutions, CamShift and Meanshift might provide all you need, as well as running a feature description and then applying homography.

I've had good luck with the MILTracker in the OpenCV contrib repository. They have a tracker interface there are quite a few different algorithms. If you compile OpenCV with that added in, you have a nice consistent API where you can pretty simply plug and play various trackers as well as write your own that conform.....that's only if you need to get pretty fancy though.

edit flag offensive delete link more


Thanks for your answer. Your suggestions make sense. However, would you be able to help me further by providing example code of how you would use hierarchies, running some type of feature descriptor, using the ROI filtering etc. I am still pretty new to opencv and am struggling to find the right sources myself. Thanks!

joljols gravatar imagejoljols ( 2015-01-20 04:20:55 -0500 )edit

Still haven't received any constructive coding help unfortunately..

joljols gravatar imagejoljols ( 2015-01-25 09:15:54 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower


Asked: 2015-01-18 08:45:45 -0500

Seen: 778 times

Last updated: Jan 19 '15