OpenCV OMX artifacts

asked 2020-04-23 02:35:43 -0500

yun gravatar image

updated 2020-04-23 22:15:29 -0500

supra56 gravatar image

I'm using hardware encoding via FFMPEG, OpenCV and OpenMAX.

If I'm using FFmpeg h264_omx as a backend for VideoWriter() than I get an image like this. Colors obviously kind of mixed.

Other backend codecs work fine. My test code:

import cv2 as cv
import time

CAPTURE_WIDTH: int = 1280
CAPTURE_HEIGHT: int = 720
VIDEO_RESOLUTION = (CAPTURE_WIDTH, CAPTURE_HEIGHT)
VIDEO_FRAMERATE: float = 5.0
VIDEO_EXTENSION: str = 'mov'
VIDEO_DURATION: int = 15
IMAGE_FREQUENCY: int = 1
FOURCC: int = cv.VideoWriter_fourcc(*'avc3')


def main():
    # cap  = cv.VideoCapture("udp://@127.0.0.1:5050", cv.CAP_FFMPEG)
    cap = cv.VideoCapture(0, cv.CAP_V4L2)
    cap.set(cv.CAP_PROP_FRAME_WIDTH, CAPTURE_WIDTH)
    cap.set(cv.CAP_PROP_FOURCC, cv.VideoWriter_fourcc(*'MJPG'))
    cap.set(cv.CAP_PROP_FRAME_HEIGHT, CAPTURE_HEIGHT)
    cap.set(cv.CAP_PROP_FPS, VIDEO_FRAMERATE)
    out = cv.VideoWriter(f'file.{VIDEO_EXTENSION}', FOURCC, VIDEO_FRAMERATE, VIDEO_RESOLUTION)
    print(cap.get(cv.CAP_PROP_FOURCC))
    start = time.time()
    frames = []
    while 1:

        _, frame = cap.read()

        if frame is None:
            print('system exit)))))')
            raise SystemExit

        # yuvj422p

        # convert to grey
        # grayFrame = cv.cvtColor(frame, cv.COLOR_BGR2GRAY)
        # grayFrame = cv.cvtColor(grayFrame, cv.COLOR_GRAY2BGR)

        frames.append(frame)
        if time.time() - start > VIDEO_DURATION:
            start_time = time.time()
            print(len(frames))
            [out.write(frame) for frame in frames]
            print(time.time() - start_time)
            out.release()
            cap.release()
        time.sleep(1 / VIDEO_FRAMERATE)


if __name__ == '__main__':
    main()

The most interesting part, ff I'll just re-encode video via the FFMpeg command line, the output will be okay - FFmpeg command

What is the workaround and where should I look?

  • Machine: Raspberry Pi 4B+
  • System: Custom Yocto Distribution (latest poky, meta-openembedded, meta-raspberrypi master branches)
  • OpenCV version: 4.1.0
  • Python version: 3.8.2
  • FFMpeg version: 4.2.2 (build with --enable-omx and --enable-omx-rpi)
edit retag flag offensive close merge delete

Comments

Try it on both 4K tv or 4K monitor. You probably use 1080p which is not HDMI. It worked for me.

supra56 gravatar imagesupra56 ( 2020-04-23 22:33:01 -0500 )edit

@supra56 you got me wrong. how is this related to output video?

yun gravatar imageyun ( 2020-04-24 04:47:33 -0500 )edit

The RPIi 4B will work on HDMI AND 4K. But not 1080p which is not HDMI. The 1080p is low frequency.

supra56 gravatar imagesupra56 ( 2020-04-24 22:30:28 -0500 )edit

I ran your code in my 4k monitor. And I see nothing wrong. But why are you using duplicated cv.VideoWriter? Just comment out cap.set(cv.CAP_PROP_FOURCC, cv.VideoWriter_fourcc(*'MJPG')). this may cause your monitor. Another thing is remove this cv.CAP_V4L2 too.

supra56 gravatar imagesupra56 ( 2020-04-29 06:02:58 -0500 )edit

@Supra There's only 1 instance of VideoWriter. My webcam can provide yuv raw video or compressed mjpeg. Raw video more than 640x480 causes the camera to hangs so I am forced to use mjpeg. And I don't have other backends for capturing video. So v4l2 is my choice.

yun gravatar imageyun ( 2020-04-30 04:46:06 -0500 )edit