Ask Your Question
5

IP camera video corruption - solutions?

asked 2015-04-13 17:49:35 -0600

Jim OSullivan gravatar image

Has anyone been able to get an HD (720p or higher) IP camera working with OpenCV consistently and reliably? If so, did you use the standard VideoCapture interface or something else?

Many OpenCV users such as here and here have reported similar problems with using OpenCV to receive video from network cameras. The typical problem involves getting a bunch of decode errors in the console as well as seeing the image corrupted as follows:

image description

Some have reported that appending '?tcp' to the camera URL fixes the problem at the cost of latency, but this fix works only sometimes.

To characterize the problem, I've run a simple OpenCV video display program under various operating systems and configurations and each of the configurations below gets decoding errors:

  • Windows, OCV 2.4.11, with WIN32 graphics, no TBB
  • Windows, OCV 2.4.11, without WIN32 graphics, with TBB
  • Windows, OCV 2.4.11, with WIN32 graphics, with TBB
  • Windows, OCV 3, with WIN32 graphics
  • Windows, OCV 3, without WIN32 graphics, with TBB
  • Ubuntu, OCV 2.4.11, with ffmpeg, TBB and graphics
  • Ubuntu, OCV 2.4.11, no ffmpeg "Couldn't open the video file."
  • RPi Raspbian, OCV 2.4.11, with TBB and graphics

For testing, I'm using a Hikvision DS-2CD2032-i 1080p color H.264 IP camera. As a control case, I used the ffplay utility of FFMPEG to play the RTSP video stream from this camera for a day and a night with no errors.

I also tested OpenCV 2.4.11 and 3.0.0 beta under Windows 7SP1 x64, and OpenCV 2.4.11 under Ubuntu and Raspbian, with and without TBB.

Every configuration using OpenCV experienced decode errors, though under Linux the errors were less frequent. If I first record the video stream from the camera to a disk file then play the video file from disk, there are no errors.

The errors seem to occur when compressed video is streamed from a camera over the network and played back via OpenCV. Also, the frequency of decode errors seems proportional to the video bit rate. Also, the errors occur even when only 3% of a quad-core 4 GHz CPU is being used for decoding/display, so it doesn't seem to be a CPU bandwidth issue. My guess is that the problem is in the interface between FFMPEG and OpenCV.

Questions: 1. My test code is below - any obvious problems? 2. If you have an HD IP camera working reliably with OpenCV, did you use FFMPEG and the VideoCapture interface or something else?

Thank you.

#include <opencv2/videoio/videoio.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <iostream>

using namespace cv;
using namespace std;

Mat frame;
string window_name = "Video Test";

int main(int argc, char *argv[])
{
    int counter = 0;
    VideoCapture capture("rtsp://192.168.1.43?tcp");
    if (!capture.isOpened())
    {
        cout << "Couldn't open the video file." << endl;
        return 1;
    }
    namedWindow(window_name, CV_WINDOW_KEEPRATIO);
    for (;;)
    {
        capture >> frame;
        counter++;
        if (frame.empty())
            break;
//        if ((counter % 20) == 0)
            imshow(window_name ...
(more)
edit retag flag offensive close merge delete

3 answers

Sort by ยป oldest newest most voted
2

answered 2015-04-16 10:40:35 -0600

Hi,

I think your code is right. Sample Java code is as below. I also tried on lots of environments like you. With Gstream ugly & good codecs on Ubuntu 12.04, H.264 streaming is working with a latency without any frame corruption.

The code is like that :

public static void main(String[] args) {

    ImageIO.setUseCache(false);
    JFrame jframe = new JFrame("ETS New Algorithm - Suzuki85");
    jframe.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    JLabel vidpanel = new JLabel();
    jframe.setContentPane(vidpanel);
    jframe.setSize(1280, 720);
    jframe.setVisible(true);

    Mat frame = new Mat();
    VideoCapture camera = new VideoCapture("rtsp://192.168.1.229:554/av0_1");
    while (true) {
        if (camera.read(frame)) {
                ImageIcon image = new ImageIcon(Mat2bufferedImage(frame));
                vidpanel.setIcon(image);
                vidpanel.repaint();
            }
    }
}

public static BufferedImage Mat2bufferedImage(Mat image) {
    MatOfByte bytemat = new MatOfByte();
    Highgui.imencode(".jpg", image, bytemat);
    byte[] bytes = bytemat.toArray();
    InputStream in = new ByteArrayInputStream(bytes);
    BufferedImage img = null;
    try {
        img = ImageIO.read(in);
    } catch (IOException e) {
        e.printStackTrace();
    } finally {
        bytemat.release();
        bytes = null;
    }
    return img;
}

But if you need to process data (for example; motion tracking, histogram equalization etc. i mean Highgui procs) you need the Mat reference and this reference can not be gotten without FFMPEG. And OpenCV which is built from source with FFMPEG ability uses CPU more than 100% on Linux.So, you can not run properly on Linux. (Some folks said that they were succedded here and was me..:) but FFMPEG Needed not GStreamer but i couldn't get a stable environment)

So, i choosed to go on Win7 64 bit platform wt. OpenCV 2.4.11. Currently i can get MJPEG streming wt. 1920x1080 resolution at 16384 bitrate from my ip camera. Its quality is near to H.264 but data cost is twice of H.264. (H.264 = 500K, MJPEG = 800K) I also downloaded source code of OpenCV2.4.11 and open the project in Visual Studio .NET. I succedded to compile it and currently i am investigating the source code at CPP side if i will be able to find anything, i may write to here.

I saw that lots of codes exist at CPP side about comparing the FFMPEG versions. Sample like that :

image description

And lastly, you are right that MPEG4 or H.264 encoded files on disk are played without any error.

IP Camera H.264 640x480 RTSP Sample on Win7 64 bit and OpenCV2.4.11 (wt. Rendering errors) image description

H.264 Encoded MP4 File 640x480 Sample on Win7 64 bit and OpenCV2.4.11 (wt. out errors) image description

I guess the problem is related to network side. If it was related to FFMPEG, the H.264 encoded file also rendered wt. errors. GStreamer on Linux can render it wt. out errors with a latency. So, I think if grapb - retrieve cylec can wait some time about being sure that the frame is gotten properly than there will be no problem. I mean i can accept 2-3 seconds delay if Mat reference will be able to be given after reading the H..264 encoded video data.

See you on ... (more)

edit flag offensive delete link more
1

answered 2016-02-24 05:43:49 -0600

recently i have solved the same problem and try to explain the steps i followed.

i updated the most recent opencv_ffmpeg.dll i renamed opencv_ffmpeg.dll to opencv_ffmpeg310.dll to use with OpenCV 3.1, also renamed same dll opencv_ffmpeg2412.dll to use with OpenCV 2.4.12

by doing that, a basic capturing frames and display became successful without problem.but still the same problem if i do some image-processing or detection causes delay between capturing frames.

to solve the second problem i used a thread to grab frames continiously and update a global Mat for processing.

here you can find my test code ( it need some improvements like using mutex and lock memory when update the Mat)

i hope the information will be useful ( sorry for my poor english )

edit flag offensive delete link more

Comments

This approach is working also at Java side. OpenCV 3.x release had fixed issues at ffmpeg side. So as sturkmen said renaming the dll through your 2.x release name and using it with threads will solve the problems. Through this you will not need to integrate libvlc or any other custom solution or external library.

The approach is like that :

0- Rename the 3.x ffmpeg dll to 2.X and replace it on OpenCV base setup folder.

1- In a class use a method only reading next frame. Simply no additional process here :

Mat image;
if (videoSignalOkay) {
caprure.read(image);
}

2- In a render thread get the bufferedimage from Mat

while (streaming) { 
Thread.sleep(5); // Give a breath for CPU
streamer.read();
}

3- Updater thread will get the same reference from class of

erohan gravatar imageerohan ( 2016-02-24 07:10:06 -0600 )edit

Thanks you ! your solution solved my problem. appreciated

Iraknam gravatar imageIraknam ( 2016-10-17 22:48:24 -0600 )edit

Where should i put this ".dll" file if i have using opencv 3.1? And is it require to cmake again or some linker option?.

kishan patel gravatar imagekishan patel ( 2018-08-02 00:42:55 -0600 )edit

i think opencv 3.1 is old. i suggest you to update the latest version

sturkmen gravatar imagesturkmen ( 2018-08-02 02:18:46 -0600 )edit
0

answered 2015-07-13 09:54:37 -0600

joecmama gravatar image

I struggled with similar issues and think I have solved some of your problems using libVLC with OpenCV. I use MS Visual Studio on Windows 7 and 8.1. Details are given here: http://answers.opencv.org/question/65932

edit flag offensive delete link more

Question Tools

4 followers

Stats

Asked: 2015-04-13 17:49:35 -0600

Seen: 17,222 times

Last updated: Feb 24 '16