Ask Your Question
1

reduce processing time of webcam capture

asked 2017-02-23 05:40:06 -0500

carton99 gravatar image

updated 2017-02-23 05:42:54 -0500

Hi, I work on real time application. My processing time of application is 25ms per frame with 9ms for the cap function of VideoCapture class.

Exemple of my code:

  • begin = clock();
  • Mat image2;
  • cap >>image2;
  • end = clock(); double
  • elapsed_ms1 = double(end - begin) /CLOCKS_PER_SEC;

I would like to know if it's possible by init option of VideoCapture or other way to reduce the processing time to get frame of webcam.

thank, CC

edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted
1

answered 2017-02-23 11:02:13 -0500

updated 2017-02-27 15:00:53 -0500

One thing you can do is you can run a separate thread for grabbing the images from the video capture object, and put that captured images in a queue, and from the main thread get a frame from the queue and do your processing.

So while the main thread is processing the image for 25ms, the grabber thread will be able to grab (25/9)=2 frames in the queue ready for the main thread to process and the main thread wont have to wait 9ms for image to be captured.

this will completely remove that 9ms time that was being added to your total time

So right now its like : grab frame(9ms)+ process(25ms)=total time (34ms)

with a separate thread running:

grabber thread:->grab(9ms)--->grab(9ms)------->grab(9ms)----->grab(9ms)->-

main thread: ---> wait(9ms)->getFrame()->--Process------(25ms)---getFrame().... total time for one process cycle is 25ms

so this way the main thread will only wait 9ms in the beginning for the first frame and wont have to wait again anymore

Update: Did some testing in python by myself to see how much it improves the processing in practical, and this is the result

import threading
import time
import Queue
import cv2
frames = Queue.Queue(5)

class ImageGrabber(threading.Thread):
    def __init__(self, ID):
        threading.Thread.__init__(self)
        self.ID=ID
        self.cam=cv2.VideoCapture(ID)
        self.cam.set(3,1280) # just to increase capture time
        self.cam.set(4,1024) # just to increase capture time
        self.runFlag=True

    def run(self):
        print "sub started"
        global frames
        while self.runFlag:
            ret,frame=self.cam.read()
            frames.put(frame)
            time.sleep(0.01)
        self.cam.release()
    def stop(self):
        self.runFlag=False



##------------------------##
## Some random processing ##
## -----------------------##

def Process(img):
    #img1=cv2.Canny(img,100,200)
    img2=cv2.blur(img,(5,5))

##------------------------##
##    without threading   ##
##------------------------##

cam=cv2.VideoCapture(0)
cam.set(3,1280) # just to increase capture time
cam.set(4,1024) # just to increase capture time
ret,img=cam.read()# to ignore the lag of first capture
sumTime=0
i=0
while i<30:
    i=i+1
    start=time.time()
    ret,img=cam.read()
    capTime=time.time()
    Process(img)
    end=time.time()
    sumTime=sumTime+end-start
    print end-start, "capture time=",capTime-start

print "avarage time without multithread=",sumTime/30    
cam.release()


##------------------------##
##     with threading     ##
##------------------------##
grabber = ImageGrabber(0)
grabber.start()
i=0
sumTime=0
while i<30:
    if(not frames.empty()):
        i=i+1
        start=time.time()
        Currframe=frames.get()
        Process(Currframe)
        end=time.time()
        sumTime=sumTime+end-start
        print end-start
print "avarage time with multithread=",sumTime/30    
grabber.stop()
grabber.join()

And this is the output

0.108999967575 capture time= 0.0929999351501
0.0569999217987 capture time= 0.0420000553131
0.0190000534058 capture time= 0.00499987602234
0.055999994278 capture time= 0.0389997959137
0.0340001583099 capture time= 0.0210001468658
0.0820000171661 capture time= 0.069000005722
0.0350000858307 capture time= 0.0199999809265
0.0660002231598 capture time= 0.0530002117157
0.0350000858307 capture time= 0.0209999084473
0.0629999637604 capture time= 0.0479998588562
0.0239999294281 capture time= 0.00499987602234
0.0440001487732 ...
(more)
edit flag offensive delete link more

Comments

Things aren't so simple... threads need to be sync because you need of a shared buffer... and you can't grab so fast because you will have buffer full soon ... and if you do the works in parallel you might delay the processor time too because OpenCV uses thread internally. See this for an accurate test and code.

pklab gravatar imagepklab ( 2017-02-24 14:00:05 -0500 )edit

Hi, you right!!!

Use thread to parallelise capture of webcam and mutex to lock Mat do not work !


1 h thinking I think is openCV functions are already parallelesed, effectively hyper-threading (intel cpu) core % usage is ambigu and give contresens of openCV optimisation.

So answer is, is not necessary to parallelise cap function because the work is already done?

carton99 gravatar imagecarton99 ( 2017-02-26 10:04:36 -0500 )edit

yes my comment was on this

pklab gravatar imagepklab ( 2017-02-27 13:00:07 -0500 )edit

@Codacus thank you to share your python code with multithread grabbing. Threads are an option, also as is in my answer but

  • timing should be evaluated in overall. For example you are comparing different average time... try to take start/stop before/after while in both cases
  • In addiction what happens with more complicated Process (this cases would use more threads by OpenCV)
  • Finally, also CPU model has a role. If CPU isn't multicore, threading might degrade performance
pklab gravatar imagepklab ( 2017-03-03 02:52:04 -0500 )edit
0

answered 2017-02-23 06:44:06 -0500

pklab gravatar image

You don't have a lot of chance to reduce time to get the frame from a cam but you can use some dead time within the read() (or >>operator) process. The method combines VideoCapture::grab() and VideoCapture::retrieve() in one call. This chain have to wait for a new frame from the cam, if the next frame isn't ready you have dead time.

You could have some improvement if you can call grab() > process prev frame > call retrieve(). Another option is to use a grab thread and processor thread... there are many example on this forum.

last but not least option is application optimization :)

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-02-23 05:40:06 -0500

Seen: 1,934 times

Last updated: Feb 27 '17