I've been struggling with this for a few months now and thought I should consolidate my efforts and "research" into this question all in one place. Mainly I am doing this to help those that have this issue or will face it in the future, so they don't have to waste valuable time.
It appears that others have struggled as well, since OpenCV uses ffmpeg library for rtsp and such, but has decoding errors for at least h264 video streaming (See link1, link2, link3 for some examples of broken images).
I have multiple IP cameras that I connected to a switch, and then need to process the video feeds through the rtsp commands that was given by the camera manufacturer: "rtsp://username:[email protected]:554" (Here, 554 is the RTSP port number assigned to the camera, which may require you to go an adjust the number by logging into the camera and accessing the camera settings)
I wanted to use my OpenCV code but could not just use: VideoCapture cap1("rtsp://username:[email protected]:554"); or, cap1.open("rtsp://username:[email protected]:554"); because of the broken video feeds from FFMPEG as explained above.
Trying to do the TCP protocol instead of UDP may have fixed the issue for others, but this is already incorporated in the latest versions of OpenCV (it was for 2.4.11. and a new update should fix it for 3.0.0; see this bug report) and did not work for me.