Apologies for my inexperience in this domain..I am trying to implement an algorithm that detects the occurrence of a particular event in real-time. The particular event is a consecutive growth of motion across 5 consecutive frames.. almost analogous to a growing sphere or beach ball.
I am able to detect the event on pre-recorded video that is in .avi format (mjpeg frames) with EmguCV (C# wrapper for OpenCV). The method I use is based on background subtraction.. outlined here https://www.pyimagesearch.com/2015/05/25/basic-motion-detection-and-tracking-with-python-and-opencv/
The problem is that the live video transport stream is usually in the format rtsp://XXX.XXX.X.XX/stream1.sdp
EmguCV on windows can't decode this h.264 stream for some reason that I am still trying to figure out ... I tried the same url using Python and OpenCV and received a non-matching transport in server reply message similar to this one https://stackoverflow.com/questions/49668941/nonmatching-transport-in-server-reply-when-cv2-videocapture-rtsp-onvif-camera - the answer didn't work for me.
I can open the rtsp URL using VLCPlayer and its corresponding C# library - from my understanding it is using ffmpeg, although I may be wrong. FFmpeg on the command line can access the stream.
EmguCV also uses ffmpeg as a backend which is why I am very confused as to why it can't open the rtsp URL.
Here is an image of the module tree when VLCPlayer opens the rtsp stream: 
. From my understanding, EmguCV doesn't use live555 or avcodec..
I've noticed that if I change the streamer configuration to use UDP or RTP rather than RTSP, EmguCV can access the h.264 URL, although the URL is now in the format rtp/udp://XXX.XXX.X.XX:XXXXX - no .sdp extension.
I would highly appreciate if someone with more experience can give me some pointers. I have a great deal to learn even though I have spent a lot of time researching this topic. In regards to the detections remaining successful would it be recommended to process H.264 frames with possible distortion or MJPEG frames?
I can't afford a delay longer than 1-2 seconds, and would ideally like to continue with the current method used to detect the event.
From my current understanding, here are the routes I can take:
1) Use RTP/UDP and process h.264 video using EmguCV - there is some distortion in the video when there is a large amount of movement.. I also receive several h264 error messages during the stream
[h264 @ 00000124f13a5080] SPS unavailable in decode_picture_timing
[h264 @ 00000124f13a5080] non-existing PPS 0 referenced
[h264 @ 00000124f13a5080] decode_slice_header error
[h264 @ 00000124f13a5080] no frame!
[h264 @ 00000124f135eac0] Missing reference picture, default is 0
[h264 @ 00000124f135eac0] decode_slice_header error
[h264 @ 00000124f13a5080] cbp too large (6929) at 11 20
[h264 @ 00000124f13a5080] error while decoding MB 11 20
[h264 @ 00000124f135eac0] top block unavailable for requested intra mode -1
[h264 @ 00000124f135eac0] error while decoding MB 3 0
[h264 @ 00000124f124e580] cbp too large (96) at 33 0
[h264 @ 00000124f124e580] error while decoding MB 33 0
[h264 @ 00000124f19940c0] top block unavailable for requested intra mode
[h264 @ 00000124f19940c0] error while decoding MB 1 1
2) Keep RTSP protocol, use libav to decode the frames and pass to EmguCV.. following this answer https://www.raspberrypi.org/forums/viewtopic.php?t=83127 .. I'm not sure if this will introduce a huge delay
3) Keep RTSP protocol, use ffmpeg to convert h.264 stream to MJPEG and access this URL instead? Again I'm not sure if this will be a feasible solution if it will introduce a great delay.
4) Use a Linux machine rather than windows and configure a gstreamer backend - not ideal
Thank you for taking the time to read this post.