neoirto's profile - activity

Hi all,

We need to work on a long duration 4K AVCHD video sequence and generate a low res preview based on that file.

We thought to first convert the 4K original video to a LOW RES PROXY file (in a separate thread using opencv_ffmpeg.dll or external ffmpeg.exe cmd), and then work on that proxy file to speed up preview.

Idealy we would like to be able to read that PROXY file with VideoCapture before the end of the conversion.

Do you think it's possible without using image sequence? What codec would you use to generate that proxy?

Tx

Hi,

I have a strange result when reading YUV420 video with gpu::VideoReader_GPU. FFmpeg on CPU give good results, but the gpu version output strange colors (especially in blue), near the true colors but not exactly the same...

cv::gpu::GpuMat             d_frame;
cv::gpu::VideoReader_GPU    d_reader("path.....");

/// then the following read() works, but colors are not exactly true...
d_reader.read(d_frame);

/// knowing that :
d_reader.dumpFormat(std::cout);

/// give the following output:
Frame Size    : 1920x1088
Codec         : H264
Chroma Format : YUV420

... instead of 1920x1080 : it appears like an old bug in FFMPEG, that was corrected since then. But the problem probably does not come from there, because the result is the same with other sources like :

Frame Size    : 1920x1080
Codec         : MPEG4
Chroma Format : YUV420

I compiled opencv 2.4.6 with CUDA and NCUVID. Is there any knowned fix for that problem ?

Thanks for your answers ;)

ENCODING OUTPUT VIDEO

• NVIDIA VIDEO CODEC SDK with several solution to encode with GPU, compatible on Win and Linux, but not Mac actually ?

• other solution is to use ffmpeg from OpenCV, which works great, but a bit slow on CPU of course... But if all my treatment is on GPU, it could be aceptable, I'll have to test that.

Tx again !

What function could be used to determine if hardware is compatible with multiple streams and how much for n ?

• do you confirm the CPU could be released after each call to a new stream ?

  • is it something related with the use of : stream.enqueueHostCallback( receive_data_from_GPU, Data_to_GPU) ?
  • and after that call CPU is awaken by receive_data_from_GPU( cv::gpu::Stream&, int status, void* Data_from_GPU ) function ? The difficulty for me at this stage is I own an ATI card only : may I choose a >= 2.0 compatibility card, or 1.3 is enough ?

• Another point : is it possible to declare a memory space on GPU, that would be directly accessible by any of the "n" streams for a "second level" of the algorithm ?

Thanks for your answer JasonInVegas !

So what I understood is :

1. DECODING INPUT

   • NVCUVID (ie NVIDIA VIDEO CODEC SDK) can be used to decode video files, but it depends on video formats and hardware compatibility,
   • other solution is to use ffmpeg from OpenCV, which works great, but a bit slow on CPU of course...

2. OpenCV + CUDA treatment :

   • To declare and use several streams on the GPU : cv::gpu::Stream stream[ n ] but only if "compute capability " of hardware is greater than 2.0 ? as this thread suggests : http://opencv-users.1802565.n2.nabble.com/Parallelling-CUDA-devices-td7580560.html you say : "most likely you will want to parameterize the stream counts (rather than hard-coding '12')", but on what criterion ?

Hi,

I'd like to know if the following CUDA pseudocode is feasible ?

• 1 dispatcher CPU thread, that will :

  -- initialize CUDA streams, saying 12 differents streams for example, each stream may run the same GPU code

  -- manage I/O frames on this CPU thread with VideoCapture/VideoWriter,

  -- for each frame :

  --- feed 1 free stream of the 12 CUDA streams in async to get the best usage of the total bandwith transfer (is async DMA transfer possible on every graphic cards or not ? >= compute capability x.x ?), with optimized struct for each data transfer
  --- release CPU until a Callback is done from the GPU : is it possible ?
  

  --- receive async resulting data from any of the 12 GPU streams : so wake up the CPU thread, that will handle the Videowriter, and send a new frame to that free GPU stream... etc ?

What low cost Nvidia card would you advise for the best results ?

I understood GPU class is "compute capability 1.3" actually, but would it be 2.0 or higher in a near future ?

Tx for your answers ;)

Hi,

You should :

• first undistort your raw image

• then apply your warpperspective()

And use the undistorted camera matrix of course ( Intrinsic_undisto ).

To get the transform Mat, you don't have to do such complex computes.

Mat Trans_to_C= (   Mat_<double>(3,3) <<
1, 0, -Cx,
0, 1, -Cy,
0, 0, fx);

Mat Intrinsic_undisto = (   Mat_<double>(3,3) <<
fx, 0, Cx,
0, fy, Cy,
0, 0, 1);

Mat 3x3_Translation_bird_eye = (    Mat_<double>(3,3) <<
1, 0, Trans_X,
0, 1, Trans_Y,
0, 0, Trans_Z);

Mat transform =  3x3_Translation_bird_eye * Intrinsic_undisto * 3x3_Rot * Trans_to_C;

I'm not sure for 3x3_Translation_bird_eye, but you should try this...

Ok, I found by myself :

Compiling opencv 2.4.5 + OCL : same result

Compiling AMD APP samples, always with Mingw : it runs... But with results like this :

Group Size specified : 256
Max Group Size supported on the kernel(writeKernel) : 128
Falling back to 128
Executing Kernels for 1 iterations
-------------------------------------------

So I suspect the actual opencv limitation to workgroup size to 256, because my GPU is a Radeon HD 5450, limited to 128.

Could you please tell me if there is any plan to add the feature to future version (2.4.6, 3.0 ??)

In a near future, will it be possible to call different OCL contexts at the same time too (from several threads) ?

Hi all,

I compiled OpenCV 2.4.3 with success, with the OpenCL support AMD SDK v2.8, with Mingw on Win7 64. Now, my first program compiles fine, but crashes when executing a simple :

ocl::oclMat OCL_Mat_Img( Mat_Img );

with the following error :

OpenCV Error: Gpu API call (CL_INVALID_BUFFER_SIZE) in openCLMallocPitch,...

There is probably something wrong in this first OpenCL test for me... For example, where should I place .cl kernels regading to my .exe ?

Thanks for your help !

Hi all,

I try to stitch 2 undistorted images (Mat_Img0 and Mat_Img1), knowing there individual orientations (Mat_RotImg0 and Mat_RotImg1). So I'd like to stitch Mat_Img1 directly on Mat_Img0. I tried the following :

double  Mat_RotImg0_DATA[4][4] = {      {(double)RotImg0_00, (double)RotImg0_01, (double)RotImg0_02, 0},
                                            {(double)RotImg0_10, (double)RotImg0_11, (double)RotImg0_12, 0},
                                            {(double)RotImg0_20, (double)RotImg0_21, (double)RotImg0_22, 0},
                                            {0,0,0,1}
                                        };
Mat         Mat_RotImg0 = Mat(4, 4, CV_64F, Mat_RotImg0_DATA );


double      Mat_RotImg1_DATA[4][4] = {      {(double)RotImg1_00, (double)RotImg1_01, (double)RotImg1_02, 0},
                                            {(double)RotImg1_10, (double)RotImg1_11, (double)RotImg1_12, 0},
                                            {(double)RotImg1_20, (double)RotImg1_21, (double)RotImg1_22, 0},
                                            {0,0,0,1}
                                        };
Mat         Mat_RotImg1 = Mat(4, 4, CV_64F, Mat_RotImg1_DATA );



////////// Find rotation between the 2 images
Mat         Mat_RotImg1_to_RotImg0 = ( Mat_RotImg1.t() * Mat_RotImg0 );



Mat         A1 = (Mat_<double>(4,3) <<
            1, 0, -cx,
            0, 1, -cy,
            0, 0,    0,
            0, 0,    1);

Mat         T = (Mat_<double>(4, 4) <<
            1, 0, 0, 0,
            0, 1, 0, 0,
            0, 0, 1, fx,
            0, 0, 0, 1);

Mat         A2 = (Mat_<double>(3,4) <<
            fx,     0,      cx,     0,
            0,      fy,     cy,     0,
            0,      0,      1,      0);

Mat         Mat_Rot_warp = A2 * ( T * ( Mat_RotImg1_to_RotImg0 * A1));

////////// Apply the stitching Mat Mat_Rot_warp
warpPerspective(    Mat_Img1, Mat_Img0, Mat_Rot_warp, Mat_image_corrected.size(), INTER_NEAREST, BORDER_CONSTANT, Scalar(0, 0, 0) );

Unfortunately, it does not give the expected result. Do you see anything I do wrong in this code ?

Thanks a lot in advance for your answers ;)