Ask Your Question
0

Streaming a webcam at 60 fps

asked 2017-06-14 04:06:13 -0600

Tachyon1979 gravatar image

updated 2017-06-20 18:29:51 -0600

hi everybody

i have a problem with capturing a webcam (logitech c922) video stream. the webcam provides a stream at 60fps at a resolution of 720p.

right now i initialize a capture convert it from BGR2toGray send the frame to an ImageView object.

here is my code.

    public void startCapture(ImageView realView, int frameWidth, int frameHeight){

    moveCoordSystemToCenter(frameWidth/2, frameHeight/2);  

    if (!this.capture.isOpened()){
        capture.open(0);

        int fourcc = VideoWriter.fourcc('M', 'J', 'P', 'G');
        capture.set(Videoio.CAP_PROP_FOURCC, fourcc);   
        capture.set(Videoio.CAP_PROP_FRAME_WIDTH,1280);
        capture.set(Videoio.CAP_PROP_FRAME_HEIGHT,720);
        System.out.println(capture.get(Videoio.CAP_PROP_FRAME_WIDTH));

            TimerTask frameGrabber = new TimerTask(){
                public void run(){

                    long time = System.currentTimeMillis();
                    Image tmp =  grabFrame();

                    Platform.runLater(new Runnable(){   
                        int Counter = 1;
                        long res = 1;

                        public void run(){
                                realView.setImage(tmp);
                                res = res + System.currentTimeMillis()-time;
                                System.out.println("Fps = " + 1000 / (res/Counter)) ;
                                Counter++;
                        }
                    });
                }
            };      
                            this.timer = new Timer();       
                            timer.scheduleAtFixedRate(frameGrabber, 0, 1); // has no effect
    }
    else
    {
        if (this.timer != null){
            this.timer.cancel();
            this.timer = null;
        }
    this.capture.release();
    }       
}

private Image grabFrame() {

    Image imageToShow = null;
    Mat frame = new Mat();

    if ( this.capture.isOpened()){
            try
            {

                this.capture.read(frame);
                if (!frame.empty()){
                    imageToShow = mat2Image(frame);
                }
            }
            catch (Exception e){
            }
    }
    return imageToShow; 
}

private Image mat2Image (Mat frame){
    MatOfByte buffer = new MatOfByte();
    Mat frameGray = new Mat();
    Imgproc.cvtColor(frame, frameGray, Imgproc.COLOR_BGR2GRAY);
    Imgcodecs.imencode(".png",frameGray, buffer);
    return new Image (new ByteArrayInputStream(buffer.toArray()));

}

i use opencv3.1 Java, Eclipse neon, JavaFx

my problem is i can´t get more than 30 fps out of this thing i Need the 60 fps for my application. to set the codec to M J P G brings up speed from 15 fps to 30fps and MJPG is the only Codec which holds my resolution all others fall back to 640 / 480

so i play with some formats of video- and Picture codecs stuff.

Jpg an Png is simular stable at 30 fps bmp is a little faster but not so stable 28- 36 fps

i already tried to set the fps via capture.set(CAP_PROP_FPS,60) which has no effect for values above 30.

I read some stuff about ffmpeg but i don´t understand how the pipeline should work.

can anyone help.

greetz

edit retag flag offensive close merge delete

Comments

i can only guess (no java here), but i'd try first to profile the mat2Image() function, which is most likely your bottleneck. do you have to use javafx here ? encoding an image to png is expensive, and your imageview also has to decode it again, seems to be pretty wasteful.

also, you might need to process at 60 fps, but do you need to draw at the same framerate ?

berak gravatar imageberak ( 2017-06-18 03:01:08 -0600 )edit

1 answer

Sort by » oldest newest most voted
0

answered 2017-06-20 18:40:00 -0600

Tachyon1979 gravatar image

updated 2017-06-20 18:53:57 -0600

After a lot of tests with ffmpeg and try to understand how i can get access to the webcam features in opencv. I found out that my cam is a so called "dshow" device which is provided from Windows Standard Webcam Driver.

So to get deeper access to the device I have to start the capture with this flag.

capture.set(Videoio.Cap_DSHOW + 0); // instead of capture.set(0);

The FourCC flag is also necessary.

    int fourcc = VideoWriter.fourcc('M', 'J', 'P', 'G');
    capture.set(Videoio.CAP_PROP_FOURCC, fourcc);

Now this line has an effect..

capture.set(Videoio.CAP_PROP_FPS,60);

After this changes i can capture with 58 fps and have almost no latency!

Hope this helps someone!

edit flag offensive delete link more

Question Tools

2 followers

Stats

Asked: 2017-06-14 04:06:13 -0600

Seen: 6,275 times

Last updated: Jun 20 '17