How to record the time of stay by detected people in a video?

edit

You are able to detect people in a frame using HOG features. In order to compute how long a person was present in a video, we need to know the first and last frames of their presence in the video and the video's frame rate.

timeOfStay = (lastFrame - firstFrame) / frameRate

If you would like to record time of stay for each person, detection is not enough. We need to able to track them. Since there is no intersection of trajectories, a simple Kalman filter would suffice. Here is a good example: http://www.morethantechnical.com/2011...

You get the first frame of detection from initializing a kalman filter and last frame when you lose track of the person of interest. OpenCV provides both frame number and frame rate and hopefully this helps with your problem.

Edit: Including information on kalman filter and object tracking

HOG provides detections in a given frame. A video has multiple frames and we now have all these different detections in each of these frames. If we expect to see only one person, then just the occurrence of that person (HOG detection) is enough to identify which of the frames that person was present. In this case, we have multiple entities. We need to associate detections in each frame with detections in the previous frame and account for addition or deletion of entities.

Here are some good tutorials on kalman filter and object tracking: http://www.mathworks.com/help/vision/... https://www.youtube.com/watch?v=FkCT_... https://www.youtube.com/watch?v=NT7nY... https://www.youtube.com/watch?v=rUgKn...

There is some C++ and OpenCV code here: https://github.com/Smorodov/Multitarg...

i tried to implement a sample code just to be a starting point for a solution of your question.

could you try it and tell me your remarks. Maybe i will try to improve it or some ideas will raise.

i tested it with 768x576.avi can be found OpenCV's \samples\data

( tracking is not implemented yet. my main idea was speeding up pedestrian detection )

for testing with OpenCV 2.4.x see Github Link

#include <opencv2/imgproc.hpp>
#include <opencv2/highgui.hpp>
#include<opencv2/objdetect/objdetect.hpp>
#include "opencv2/video/background_segm.hpp"

using namespace cv;
using namespace std;

int main( int argc, char** argv )
{
    char* filename = argc >= 2 ? argv[1] : (char*)"768x576.avi";
    VideoCapture capture( filename );

    //HOGDescriptor hog;
    //hog.setSVMDetector(hog.getDefaultPeopleDetector());

    HOGDescriptor hog( Size( 48, 96 ), Size( 16, 16 ), Size( 8, 8 ), Size( 8, 8 ), 9, 1, -1,
                       HOGDescriptor::L2Hys, 0.2, false, cv::HOGDescriptor::DEFAULT_NLEVELS);
    hog.setSVMDetector( HOGDescriptor::getDaimlerPeopleDetector() );

    Ptr<BackgroundSubtractor> bgS = createBackgroundSubtractorMOG2();
    Mat frame,output;

    while(true)
    {
        capture.read(frame);
        if (!frame.data)
            return 0;
        if( frame.cols > 800 )
            resize( frame, frame, Size(), 0.5, 0.5 );

        bgS->apply(frame, output);
        erode(output,output,Mat());

        // Find contours
        vector<vector<Point> > contours;
        findContours( output, contours, RETR_LIST, CHAIN_APPROX_SIMPLE );

        for ( size_t i = 0; i < contours.size(); i++)
        {
            Rect r = boundingRect( contours[i] );
            if( r.height > 80 & r.width < r.height )
            {
                r.x -= r.width / 2;
                r.y -= r.height / 2;
                r.width += r.width;
                r.height += r.height;
                r = r & Rect( 0, 0, frame.cols, frame.rows );

                Mat roi;
                cvtColor( frame( r ), roi, COLOR_BGR2GRAY);

                std::vector<Rect> rects;

                if( roi.cols > hog.winSize.width & roi.rows > hog.winSize.height )
                    hog.detectMultiScale( roi, rects);

                for (size_t i=0; i<rects.size(); i++)
                {
                    rects[i].x += r.x;
                    rects[i].y += r.y;

                    rectangle( frame, rects[i], Scalar( 0, 0, 255 ), 2 );
                }
            }
        }

        imshow("display", frame);
        if(waitKey(30)==27)
        {
            break;
        }
    }
    return 0;
}

In order to achieve this in more efficent way, you need to have some kind of tracking enabled in you code. Once you have tracker, tracker will assign id to the detected person and then you can pretty much start a timer and keep counting against that person id. This way you also dont need to call your detection in each frame and you can improve some FPS.

Here, have a look at this, although its in C++ but you might get some basic idea: https://www.youtube.com/watch?v=VZjay...