Ask Your Question
3

OpenCV 3.0.0 - process new frame callback

asked 2016-01-18 23:17:10 -0500

hyder gravatar image

Hi, I am trying to implement an algorithm which requires processing a new frame as soon as it is available from the camera. The polling method cannot be used because a lot of serial commands are to be sent after processing. Rather I want to use some interrupt based method which notifies me as a new frame becomes available. I've heard that Callback function based implementation was present in OpenCV beta 5 release which would be called automatically on the arrival of a new frame and then I could implement the processing algorithm in that callback, but I can't find it in the 3.0.0. Is anything of such sort available in 3.0.0?

I have searched for the problem and found this link:link but it is fairly old.

So my question is whether such callback functionality is available in 3.0.0 or should I use some other method like multi threading?

I am sorry if I can't explain it very well. I am fairly new to openCV. Please feel free to ask if any additional information is required.

Thanks.

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
3

answered 2016-01-24 12:51:45 -0500

pklab gravatar image

cv::VideoCapture::read returns the frame as soon as it's available. With a simple loop grab+process you'll achieve exactly what you are looking for. Short hints:

  • If your processing time is always faster than 1/FPS than a simple grab+process loop works fine without threads too.
  • If sometime your processing time is slower than 1/FPS than you can use a grabbing thread, a processing thread and a shared queue (check here for an example)
  • If your processing time is always slower than 1/FPS than you have to reduce FPS or improve processing performance.

Details:

On windows, cv::VideoCapture uses internal callback with DSHOW, VFW and Microsoft Media Foundation. This means that cv::VideoCapture::read waits for the next frame as provided camera FPS. It waits (in idle state) for a signal from the callback function. Check the code for DSHOW

Duration of cv::VideoCapture::read can change from close to zero (if a frame is ready) up to 1/FPS (if a frame has just been grabbed).

Results from the code below shows that loop time (grab+process) will not change if processing time < 1/FPS... when processing time increases, grabbing time decreases because grabbing will waits less time for a new frame:

Setting wanted fps to 25.0
Measuring real fps...: 19.2fps
    Max processing time: 1/19.2 = 52.1ms

Start processing time tests...
Processing: 0.0ms   Grabbing: 46.8ms    Total: 46.8ms
Processing: 10.0ms  Grabbing: 35.4ms    Total: 45.4ms
Processing: 20.0ms  Grabbing: 26.8ms    Total: 46.8ms
Processing: 30.0ms  Grabbing: 16.8ms    Total: 46.8ms
Processing: 40.0ms  Grabbing: 6.6ms     Total: 46.6ms
Processing: 50.0ms  Grabbing: 4.1ms     Total: 54.1ms

when processing > 1/fps :

Processing: 60.0ms  Grabbing: 3.6ms Total: 63.6ms  <- we are loosing frames

The test code:

#define SimulateProcessing(_processingTimeMs)  \
    int64 _start= getTickCount();  \
    int64 _end = _start + _processingTimeMs * getTickFrequency() / 1000.;  \
    while (getTickCount() <= _end) {double d = 2*CV_PI;} 

int Main_GrabbingMeasureTimes()
{
    int camId = 0;                //the default camera
    //int driver = CV_CAP_VFW;    //use VFW driver
    //int driver = CV_CAP_MSMF;   //use MSMF driver
    int driver = CV_CAP_DSHOW;    //use DSHOW
    double fps = 25.0;            //wanted fps
    cv::Mat frame;
    int64 grabStart, grabSum, procStart, procSum;
    int cnt, maxCnt = 50;
    double procTestTime,procTestTimeMax;
    double procTime, grabTime;
    std::clog.precision(1);
    clog.setf(std::ios::fixed, std::ios::floatfield);

    //OPEN THE CAMERA
    VideoCapture cap(camId + driver);
    if (!cap.isOpened()) return -1;
    cap >> frame;                    //grab to prepare memory

    //--------------
    //MEASURE REAL FPS
    clog << endl << "Setting wanted fps to " << fps;
    cap.set(CV_CAP_PROP_FPS, fps);   //set grabbing fps
    fps = cap.get(CV_CAP_PROP_FPS);  //read fps from cam
    clog << endl << "Measuring real fps...";
    for (cnt = 0,grabSum = 0; cnt < maxCnt; cnt++) {
        grabStart = getTickCount();
        cap >> frame;
        grabSum += (getTickCount() - grabStart);
    }
    procTestTimeMax = 1000.0 * grabSum / getTickFrequency() / cnt;
    clog << ": " << 1000. / procTestTimeMax << "fps" << endl
        << "\tMax processing time: " << procTestTimeMax << "ms" << endl;

    //--------------------------------------------
    //MEASURE LOOP DURATION VS PROCESSING DURATION
    clog << "\nStart processing time tests...\n";
    //same loop with increasing processing time
    for (procTestTime = 0; procTestTime < procTestTimeMax + 10; procTestTime +=10) {
        //N frames for average
        procSum  = 0;
        for (cnt ...
(more)
edit flag offensive delete link more

Comments

@pklab, Thank you very much for a very detailed answer! I just wanted to get rid of the loop waiting process to get the next frame but as you said if my processing time (in my case 10-15ms) is greater than 1/FPS (which is 25 fps in my case giving me a time of 40ms for grabbing+processing), then there is no need for interrupt/callback based method and polling would be better and it would not drain the battery while waiting in a loop for the next frame, right?

hyder gravatar imagehyder ( 2016-01-25 01:44:39 -0500 )edit
1

If your processing time (your case 10-15ms) is less than 1/FPS (your case 40ms) you can use simple plain grab than process loop.

pklab gravatar imagepklab ( 2016-01-25 03:12:10 -0500 )edit

I don't understand your other points.The communication scheme is your_application>OpenCV>filter>driver>cam. The filter layer (eg. DSHOW) uses a waiting method (DSHOW uses WaitForSingleObject) to wait something from the driver. Library based wait functions use to poll from a list of events/messages. The list is application based and its length not change the CPU load or the battery needs. WaitForSingleObject uses ~0% CPU

The communication camera<->driver depends from manufacturer. Some cameras have interrupts and good power management some others require polling by the driver. BTW, from application, you could configure this communication only if the driver has its own dedicated functions.

pklab gravatar imagepklab ( 2016-01-25 03:26:43 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2016-01-18 22:54:24 -0500

Seen: 2,551 times

Last updated: Jan 24 '16