psykomusic's profile - activity

Hi, I have the same problem, i try to read an h264 stream from ipcamera (local network) and process motion detection. With the object VideoCapture the stream freeze after 20-60 secondes. So I want replace the VideoCapture objet by, ffmpeg or vlc, but I need a cv::Mat objet to process my motion detection. Do you have an idea hot to do it ? Thx, Psykomusic

Hi everybody !

I want to read stream from ipCamera in h264 1920*1072 and process it using Opencv.

I tried the easy way with videoCapture("rtp://myadress") but the image is very bad and i have error like : "error while decoding : MB...".

So I want to use libVlc to read the stream and access the data with openCV. I find this post and I use his code : http://answers.opencv.org/question/56... I change the size of the data for matching mine 1920*1072. But I don't really understand this code. But i have always the same console logs :

• [rawvideo @ 0x7fe04c2cd440] AVFrame.format is not set
• [rawvideo @ 0x7fe04c2cd440]AVFrame.width or height is not set
• MediaPlayerPositionChanged
• MediaPlayerTimeChanged 0 ms

And i have in my file log:

• rtp debug: ignoring late packet (sequence: 27130)
• rtp debug: ignoring late packet (sequence: 27131)
• packetizer_h264 warning: waiting for SPS/PPS
• rtp debug: ignoring late packet (sequence: 27132) and no image is shown.

using namespace cv;
    using namespace std;

struct VideoDataStruct {
  int param;
};

int done = 0;
libvlc_media_player_t *mp;
unsigned int videoBufferSize = 0;
uint8_t *videoBuffer = 0;

 void cbVideoPrerender(void *p_video_data, uint8_t **pp_pixel_buffer, int size) {
// Locking
  if (size > videoBufferSize || !videoBuffer)
  {
    printf("Reallocate raw video buffer\n");
    free(videoBuffer);
    videoBuffer = (uint8_t *) malloc(size);
    videoBufferSize = size;
  }

   *pp_pixel_buffer = videoBuffer;
}

 void cbVideoPostrender(void *p_video_data, uint8_t *p_pixel_buffer, int width, int height, int pixel_pitch, int size, int64_t pts) {
     // Unlocking
     //CloseHandle(hMutex);
 }

 static void handleEvent(const libvlc_event_t* pEvt, void* pUserData) {
   libvlc_time_t time;
   switch(pEvt->type)
   {
      case libvlc_MediaPlayerTimeChanged:
          time = libvlc_media_player_get_time(mp);
          printf("MediaPlayerTimeChanged %lld ms\n", (long long)time);
          break;
      case libvlc_MediaPlayerEndReached:
          printf ("MediaPlayerEndReached\n");
          done = 1;
        break;
       default:
          printf("%s\n", libvlc_event_type_name(pEvt->type));
    }
}

int main(int argc, char* argv[]) {

// VLC pointers
libvlc_instance_t *inst;
libvlc_media_t *m;
void *pUserData = 0;
VideoDataStruct dataStruct;

// VLC options
char smem_options[1000];
// RV24
sprintf(smem_options
    , "#transcode{vcodec=RV24}:smem{"
     "video-prerender-callback=%lld,"
     "video-postrender-callback=%lld,"
     "video-data=%lld,"
     "no-time-sync},"
    , (long long int)(intptr_t)(void*)&cbVideoPrerender
    , (long long int)(intptr_t)(void*)&cbVideoPostrender
    , (long long int)(intptr_t)(void*)&dataStruct
);

const char * const vlc_args[] = {
          "-I", "dummy",            // Don't use any interface
          "--ignore-config",        // Don't use VLC's config
          "--extraintf=logger",     // Log anything
          "--verbose=2",            // Be verbose
          "--sout", smem_options    // Stream to memory
           };

// We launch VLC
inst = libvlc_new(sizeof(vlc_args) / sizeof(vlc_args[0]), vlc_args);

/* Create a new item */
//m = libvlc_media_new_path(inst, "outputDiff.avi");
    m= libvlc_media_new_location(inst, "rtp://myadress:myport");
/* Create a media player playing environement */
mp = libvlc_media_player_new_from_media (m);

libvlc_event_manager_t* eventManager = libvlc_media_player_event_manager(mp);
libvlc_event_attach(eventManager, libvlc_MediaPlayerTimeChanged, handleEvent, pUserData);
libvlc_event_attach(eventManager, libvlc_MediaPlayerEndReached, handleEvent, pUserData);
libvlc_event_attach(eventManager, libvlc_MediaPlayerPositionChanged, handleEvent, pUserData);

libvlc_video_set_format(mp, "RV24", VIDEOWIDTH, VIDEOHEIGHT, VIDEOWIDTH* 3 );

/* play the media_player */
libvlc_media_player_play (mp);

 while(1)
 {
     if(videoBuffer)            // Check for invalid input
     {
         // CV_8UC3 = 8 bits, 3 chanels
         Mat img = Mat(Size(VIDEOWIDTH, VIDEOHEIGHT), CV_8UC3, videoBuffer);
         cvtColor(img, img, CV_RGB2BGR);
         namedWindow("Display window", WINDOW_AUTOSIZE);    // Create a window for display.
         imshow("Display window", img);     // Show our image inside it.
     }
 }
 libvlc_release (inst);
}
 
Thank you guys.
Bye
 

Hi, @ClementD, @ThorbjornSomod I want to do the same thing but i don't understand how to do it. A sample code will be useful

Hi, I don't know if your question still unanswered, but you forgot to release the videowritter. After your while you can write :

rec.release();

I resolve the error by replace the fourcc by 0x00000021

Yeah, it's a good idea ! I tried the result is less good than the optical flow. In fact my camera is static and I want record when animals or objects comes in the field, and I want to see the object moves. With your method the sum of the diff is not very high when the object moves that's why the threshold is lot more complicated and the result bad. Your way is a lot faster (1min to 6sec for a 2min video) and I'll continue to invistigate.

Thank you for your answer. :) I don't want to change for .avi, because mp4 + h264 seems to be the lightest, it's strange because i have this error but the file is written and complete. Psykomusic

Thank you for your answer. Indeed the dense optical flow seems to be overkill for the thing i want to do. Maybe i'll use 1616 block instead of 55. Psykomusic

Thank you for your answers. Indeed the dense optical flow seems to be overkill for the thing i want to do. Maybe i'll use 1616 block instead of 55. I don't want to change for .avi, because mp4 + h264 seems to be the lightest, it's strange because i have this error but the file is wititen and complete. Psykomusic

Hi guys ! I did a small code to record the moving part of a video. In fact i open a video, I process the opticalflow and I threshold the norm of the vector. When their is enough motion i record the frame in a videowritter. The code do what i want but I find it very slow, like 2min for 2min video. I found this sort of problem on this link http://stackoverflow.com/questions/28... but I'm already using ffmpeg with opencv. I don't know if it related but at the begining of the run an error occured (but no crash).

OpenCV: FFMPEG: tag 0x34363258/'X264' is not supported with codec id 28 and format 'mp4 / MP4 (MPEG-4 Part 14)' OpenCV: FFMPEG: fallback to use tag 0x00000021/'!???'

Because code is better than explanation:

int main ( argc, const char** argv)
{
    VideoCapture cap("../Video/6880.mp4");
    int fourcc = CV_FOURCC('X','2','6','4');
    int fps = cap.get(CV_CAP_PROP_FPS);
    Size S = Size((int) cap.get(CAP_PROP_FRAME_WIDTH),    // Acquire input size
           (int) cap.get(CAP_PROP_FRAME_HEIGHT));
    VideoWriter out("output.mp4",fourcc, fps ,S, true);
    Mat flow;
    UMat  flowUmat, prevgray;
    bool catchframe;
    for (;;)
    {
        bool Is = cap.grab();
        double millis = cap.get(CV_CAP_PROP_POS_MSEC); // capture time in frame
        if (Is == false)
        {
            cout << "Video Capture Fail" << endl;
            break;
        }
        else 
        {
           Mat img ;
           millis /= 1000;
           // capture frame from video file
           cap.retrieve(img, CV_CAP_OPENNI_BGR_IMAGE);
           resize(img, img, Size(640, 480));
           // save original for later
           img.copyTo(original);
           cvtColor(img, img, COLOR_BGR2GRAY);
           if (prevgray.empty() == false ) 
           {
                // calculate optical flow
                calcOpticalFlowFarneback(prevgray, img, flowUmat, 0.4, 1, 12, 2, 8, 1.2, 0);
                // copy Umat container to standard Mat
                flowUmat.copyTo(flow);
                catchframe = false;
                // By y += 5, x += 5 you can specify the grid
                for (int y = 0; y < original.rows; y += 5) 
                {
                    for (int x = 0; x < original.cols; x += 5)
                    {
                                const Point2f flowatxy = flow.at<Point2f>(y, x) * 10;
                                double norm = sqrt((flowatxy.x*flowatxy.x + flowatxy.y*flowatxy.y) * (flowatxy.x*flowatxy.x + flowatxy.y*flowatxy.y)  );
                                if (norm > 2000)
                                {
                                    catchframe = true;
                                }
                     }
                }
                if (catchframe)
                {
                    timeRejected += millis;
                    putText(original, format("%4.3f", millis),Point(20,20), FONT_HERSHEY_PLAIN, 1.0, CV_RGB(0,255,0), 2.0);
                    resize(original, original, S);
                    out.write(original);
                }
                img.copyTo(prevgray);
            }
            else                                 // fill previous image in case prevgray.empty() == true
                img.copyTo(prevgray);
         }
    }
    out.release();
}

My Question is, why my code is very slow with an intel I7, and how can I resolve the error ?