Changing the fps of CSRT or Medianflow trackers to same as KCF

asked 2018-10-23 07:04:56 -0500

meera_c gravatar image

I am using OpenCV 3.4.1 tracking classes in C++ Visual Studio 2015. Among the 8 trackers available in OpenCv, I am using mainly KCF,CSRT and MEDIANFLOW and the fps of these tracking algorithms are 30,4,4 respectively. I would like to know whether I can change the fps of these so that the videos are processed faster. Anybody knows how to increase the fps of CSRT/MEDIANFLOW to 30(same as KCF)?

In my code, KCF is faster than CSRT and MEDIANFLOW , but the tracking performance/accuracy is not as good as CSRT. But the processing speed of KCF is really good, because of its high fps.So, I need to acquire this speed in CSRT. What are the possible ways by which I can achieve this. My code part is given below:

    #include <opencv2/opencv.hpp>
    #include <opencv2/tracking.hpp>
    #include <iostream>
    #include <string>
    #include <iomanip>
    #include <map>

    using namespace std;
    using namespace cv;

    vector<string> trackerTypes = { "BOOSTING", "MIL", "KCF", "TLD", "MEDIANFLOW", "GOTURN", "MOSSE", "CSRT" };

    // create tracker by name
    Ptr<Tracker> createTrackerByName(string trackerType)
        Ptr<Tracker> tracker;
        if (trackerType == trackerTypes[0])
            tracker = TrackerBoosting::create();
        else if (trackerType == trackerTypes[1])
            tracker = TrackerMIL::create();
        else if (trackerType == trackerTypes[2])
            tracker = TrackerKCF::create();
        else if (trackerType == trackerTypes[3])
            tracker = TrackerTLD::create();
        else if (trackerType == trackerTypes[4])
            tracker = TrackerMedianFlow::create();
        else if (trackerType == trackerTypes[5])
            tracker = TrackerGOTURN::create();
        else if (trackerType == trackerTypes[6])
            tracker = TrackerMOSSE::create();
        else if (trackerType == trackerTypes[7])
            tracker = TrackerCSRT::create();
        else {
            cout << "Incorrect tracker name" << endl;
            cout << "Available trackers are: " << endl;
            for (vector<string>::iterator it = trackerTypes.begin(); it != trackerTypes.end(); ++it)
                std::cout << " " << *it << endl;
        return tracker;

    int main(int argc, char * argv[])
        const int N = 2;
        double xcordinate = 0;
        double ycordinate = 0;
        double Xvalue = 0;
        double Yvalue = 0;
        double firstFrameY[10] = {};
        double remainFrameY[10] = {};
        double firstFrameX[10] = {};
        double remainFrameX[10] = {};

        string trackerType = "CSRT";
        cout << "The Selected tracker is " << trackerType << endl;

        string videoPath = "SS-100_random11sec_Trim.mp4";
        vector<Rect> bboxes;
        Mat frame;
        cv::VideoCapture cap(videoPath);

        cap >> frame;
        bool showCrosshair = true;
        bool fromCenter = false;

        cout << "Enter the value of X-cordinate\n";
        cin >> xcordinate;
        cout << "Enter the value of Y-cordinate\n";
        cin >> ycordinate;
        cv::selectROIs("MultiTracker", frame, bboxes, showCrosshair, fromCenter);

        if (bboxes.size() < 1)
            return 0;

        Ptr<MultiTracker> multiTracker = cv::MultiTracker::create();

        for (int i = 0; i < bboxes.size(); i++)
            multiTracker->add(createTrackerByName(trackerType), frame, Rect2d(bboxes[i]));

        cout << "Started tracking, press ESC to quit." << endl;

        int m = 0;

        while (cap.isOpened())

            cap >> frame;
            if (frame.empty()) break;
            for (unsigned i = 0; i < multiTracker->getObjects().size(); i++)
                rectangle(frame, multiTracker->getObjects()[i], colors[i], 2, 1);
                bboxes[i] = multiTracker->getObjects()[i];
                //cout << "The Co-ordinates: " << bboxes[i] << endl;
                outputFile << std::left << bboxes[i].x <<',' << std::right << bboxes[i].y << '\n';  

            if (m == 0)  //for saving the reference values
                for (int i = 0; i< N; i++)
                    firstFrameY[i] = bboxes[i].y;
                    remainFrameY[i] = bboxes[i].y;
                    firstFrameX ...
edit retag flag offensive close merge delete


I have closed the previous question and created this new one with detailed explanation. Also, there was no proper answer for the previous question on this.

meera_c gravatar imagemeera_c ( 2018-10-23 07:28:25 -0500 )edit

meera_c, please don't do that, since you're removing previous answers / comments this way, and anyone has to start all over.

berak gravatar imageberak ( 2018-10-23 07:29:45 -0500 )edit

@meera_c You can close a question if nobody writes an answer or a comment. This question cannot be closed

LBerger gravatar imageLBerger ( 2018-10-23 09:12:36 -0500 )edit

If you consider each tracking method as a black box, some possible solutions,

  • change the hardware in order to make CSRT performs at ~30 fps and KCF at ~240 fps (if the number of CPU cores, CPU frequency scale linearly on the performance)
  • reduce the image resolution in order to make CSRT performs at ~30 fps and KCF at ~240 fps (if it scales linearly), the tracking accuracy should be degraded on low resolution image
  • check that OpenCV is built in Release mode, O3, etc.

Otherwise, it means dig into the source code of CSRT and try to optimize it (if it is possible) to match the performance of KCF.

Note that the reverse is much easier of course. You can degrade the performance of KCF to match the one of CSRT...

Eduardo gravatar imageEduardo ( 2018-10-23 09:27:28 -0500 )edit

@Eduardo Thank you so much for your ideas. I have few doubts about the suggestions.

  • What do you mean by changing the hardware? I am using already recorded videos in my code,not the realtime videos. I mean I don't have any hardware connected.

  • I tried it in Maximium speed(O2) and full optimization(o3), but the results are the same. CSRT is slower than KCF. It took around 34 seconds to process the 11 sec video, in CSRT. Whereas if i use KCF, it took only 16 seconds which is what I want to achieve.

  • can you point me which file contains the CSRT source code. I couldn't find anything useful so far in my search.

My motto is to achieve a high speed tracking, so degrading the performance of KCF is not an option for me. I just want the CSRT to perform at a high ...(more)

meera_c gravatar imagemeera_c ( 2018-10-23 10:36:01 -0500 )edit
  • I tried changing the video resolution of the video using the set property , but its not working.Below given is the code I used for changing the resolution and fps. Am i doing something wrong?

    string trackerType = "CSRT"; string videoPath = "SS-100_random11sec_Trim.mp4"; // 1980x1080 resolution vector<rect> bboxes; Mat frame; cv::VideoCapture cap(videoPath); cap.set(CV_CAP_PROP_FRAME_WIDTH, 320); cap.set(CV_CAP_PROP_FRAME_HEIGHT, 240); cap.set(CV_CAP_PROP_FPS,10); // changing to 10 just to check whether it works double wid = cap.get(CV_CAP_PROP_FRAME_WIDTH); double heig = cap.get(CV_CAP_PROP_FRAME_HEIGHT); cout << "Changed width nd height is:" << wid <<heig; output="" is:="" 1980="" &amp;="" 1080="" double="" fpsnew="cap.get(CV_CAP_PROP_FPS);" 30<="" p="">

meera_c gravatar imagemeera_c ( 2018-10-23 10:38:36 -0500 )edit

@meera_c , there are params to tweak but to know , what you're doing, you have to read the paper

i also don't think, that changing the resolution will help, most trackers (e.g. CSRT) don't look at the whole image, but only at some region around the current box

berak gravatar imageberak ( 2018-10-24 02:34:12 -0500 )edit

I am using already recorded videos in my code,not the realtime videos.

This is offline processing. The computation time is irrelevant in the tracking accuracy if you are processing each frame. With a real camera, you will have some frame drop if the computation time is higher than the acquisition time. Here with a video, unless you are using a separate thread to read the video, you should not lose frame.

By hardware, I meant the model of CPU for instance.

Eduardo gravatar imageEduardo ( 2018-10-24 06:07:05 -0500 )edit