H.264 or MJPEG for event detection in live stream? RTP/UDP or RTSP for accessing stream and passing frame to OpenCV?

edit

Apologies for my inexperience in this domain..I am trying to implement an algorithm that detects the occurrence of a particular event in real-time. The particular event is a consecutive growth of motion across 5 consecutive frames.. almost analogous to a growing sphere or beach ball.

I am able to detect the event on pre-recorded video that is in .avi format (mjpeg frames) with EmguCV (C# wrapper for OpenCV). The method I use is based on background subtraction.. outlined here https://www.pyimagesearch.com/2015/05...

The problem is that the live video transport stream is usually in the format rtsp://XXX.XXX.X.XX/stream1.sdp

EmguCV on windows can't decode this h.264 stream for some reason that I am still trying to figure out ... I tried the same url using Python and OpenCV and received a non-matching transport in server replymessage similar to this one https://stackoverflow.com/questions/4... - the answer didn't work for me.

I can open the rtsp URL using VLCPlayer and its corresponding C# library - from my understanding it is using ffmpeg, although I may be wrong. FFmpeg on the command line can access the stream.

EmguCV also uses ffmpeg as a backend which is why I am very confused as to why it can't open the rtsp URL.

Here is an image of the module tree when VLCPlayer opens the rtsp stream: . From my understanding, EmguCV doesn't use live555 or avcodec..

I've noticed that if I change the streamer configuration to use UDP or RTP rather than RTSP, EmguCV can access the h.264 URL, although the URL is now in the format rtp/udp://XXX.XXX.X.XX:XXXXX - no .sdp extension.

I would highly appreciate if someone with more experience can give me some pointers. I have a great deal to learn even though I have spent a lot of time researching this topic. In regards to the detections remaining successful would it be recommended to process H.264 frames with possible distortion or MJPEG frames?

I can't afford a delay longer than 1-2 seconds, and would ideally like to continue with the current method used to detect the event.

From my current understanding, here are the routes I can take:

1) Use RTP/UDP and process h.264 video using EmguCV - there is some distortion in the video when there is a large amount of movement.. I also receive several h264 error messages during the stream

[h264 @ 00000124f13a5080] SPS unavailable in decode_picture_timing [h264 @ 00000124f13a5080] non-existing PPS 0 referenced [h264 @ 00000124f13a5080] decode_slice_header error [h264 @ 00000124f13a5080] no frame! [h264 @ 00000124f135eac0] Missing reference picture, default is 0 [h264 @ 00000124f135eac0] decode_slice_header error [h264 @ 00000124f13a5080] cbp too large (6929) at 11 20 [h264 @ 00000124f13a5080] error while decoding MB 11 20 [h264 @ 00000124f135eac0] top block unavailable for requested intra mode -1 [h264 @ 00000124f135eac0] error while decoding MB 3 0 [h264 @ 00000124f124e580] cbp too large (96) at 33 0 [h264 @ 00000124f124e580] error while decoding MB 33 0 [h264 @ 00000124f19940c0 ... (more)