The first possible reason is data raise. If I understand correctly you use some time demand algorithm to process frame from camera. So, camera rewrite your mat faster then you can process it. Some parts of frame belongs to already processed frame and some parts belong to new frame from camera.
The second possible reason is invalid data access due to access to invalid submats of current frame of some operations with incompatible data types.
