how to stream h264 video with rtsp in opencv- partially answered

edit

Blockquote

Hi, I have had similar problems (and solved them) as I'll explain below. The approach to solve it, is nearly identical. My O.S is Windows 7 64 bits and I was using VC++2013.

• When streaming H264 video from IP cameras using RTSP with OpenCV , the program crashes after some minutes without a clear reason (tested it with OpenCV 3 and OpenCV 2.4.9...).

• Tried to use FFMPEG, but compiling FFMPEG was quite hard on Windows. I could not compile it correctly following the instructions I've found.

The solution: I tried libVLC and got it working for 32bits EXEs and 64 bits EXEs using:

• DLLs and plugins from VLC player 2.2.1 (if you want to generate a 64 bit EXE use the DLLs from VLC 2.2.1 64 bits, if you want to generate a 32 bit EXE use the DLLs from VLC 2.2.1 32 bits).

• .lib files and .h files for libVLC 32bits and 64 bits can be found here: https://github.com/RSATom/libvlc-sdk

• I created a simple Console project following this tutorial (https://wiki.videolan.org/LibVLC_Samp...) and added some modifications. (See the full code below). Notice the code does not use OpenCV at all, but once you got the frames, you can add OpenCV to run any process on them.

I would love to see a better RTSP support in OpenCV because everything would be easier but this is the only way I've found to solve the crashes. Please, check the code because it is a fast try...it could contain bugs.

#include <iostream>
#include <stdio.h>
#include <stdint.h>
#include <math.h>
#include <stdlib.h>
#include <assert.h>

#include "SDL.h"
#include "SDL_mutex.h"
#include "vlc/vlc.h"

#define WIDTH 800
#define HEIGHT 600

#define VIDEOWIDTH 320
#define VIDEOHEIGHT 240

struct context {
    SDL_Renderer *renderer;
    SDL_Texture *texture;
    SDL_mutex *mutex;
    int n;
};

// VLC prepares to render a video frame.
static void *lock(void *data, void **p_pixels) {

    struct context *c = (context *)data;

    int pitch;
    SDL_LockMutex(c->mutex);
    SDL_LockTexture(c->texture, NULL, p_pixels, &pitch);

    return NULL; // Picture identifier, not needed here.
}

// VLC just rendered a video frame.
static void unlock(void *data, void *id, void *const *p_pixels) {

    struct context *c = (context *)data;

    uint16_t *pixels = (uint16_t *)*p_pixels;

    // We can also render stuff.
    int x, y;
    for (y = 10; y < 40; y++) {
        for (x = 10; x < 40; x++) {
            if (x < 13 || y < 13 || x > 36 || y > 36) {
                pixels[y * VIDEOWIDTH + x] = 0xffff;
            }
            else {
                // RV16 = 5+6+5 pixels per color, BGR.
                pixels[y * VIDEOWIDTH + x] = 0x02ff;
            }
        }
    }

    SDL_UnlockTexture(c->texture);
    SDL_UnlockMutex(c->mutex);
}

// VLC wants to display a video frame.
static void display(void *data, void *id) {

    struct context *c = (context *)data;

    SDL_Rect rect;
    rect.w = VIDEOWIDTH;
    rect.h = VIDEOHEIGHT;

    // display the video in a random location
    rect.x = (int)((1. + .5 * sin(0.03 * c->n)) * (WIDTH - VIDEOWIDTH) / 2);
    rect.y = (int)((1. + .5 * cos(0.03 * c->n)) * (HEIGHT - VIDEOHEIGHT) / 2);

    // display the video in the top left corner ...