Losing alpha channel when passing from Unity to OpenCV?

asked 2016-12-20 13:01:32 -0600

Nikolaos gravatar image

updated 2017-08-22 06:51:48 -0600

Hello, I'm trying to develop an educational application using Unity3d and OpenCV (c++ Dll). My problem is that when I pass an image from unity encoded in PNG (in BGRA32 textureFromat) to OpenCV, if i save the image through OpenCV it looks like alpha channel is removed. Alpha channel is critical in some 'heavy' OpenCV-side operations for my application and this removal is disastrous.

This is what i get by saving the image from Unity (there is a transparent border) before sending it to OpenCV dll image description

This is what i get by saving the image from OpenCV image description

Here is the relative portion of the c++ code

public void saveIncomingImageArray(uchar * imMedium, int imMediumLength) {

         std::vector<uchar> vectordataImMedium(imMedium, imMedium + imMediumLength);

     cv::Mat data_mat_medium(vectordataImMedium, true);

     mediumImage = cv::imdecode(data_mat_medium, CV_LOAD_IMAGE_UNCHANGED);

     cv::imwrite("From Opencv1_medium.png", mediumImage, compression_params);

}

compression_params are defined just before the saveIncomingImageArray function
and here is the c# code respectful

//Create a texture2D to pass data from rendertexture with BGRA32 format

    Texture2D textMedium = new Texture2D(rtMedium.width, rtMedium.height, TextureFormat.BGRA32, false);

    textMedium.alphaIsTransparency = true;

    textMedium.Apply();

//fill texture2D with data from rendertexture - function defined in my code elsewhere

    RenderTextureToTexture2D(ref rtMedium, ref textMedium);

//create the array to pass to OpenCV

   byte[] imageMedium = textMedium.EncodeToPNG();

I think I need some help or some guidance, Thank you.

edit retag flag offensive close merge delete

Comments

First here it's an opencv forum (read faq) it is difficult to help you with third party lib or wrappers and c#.

I have tested :

Mat pngimg = imread("c:/Users/Laurent.PC-LAURENT-VISI/Desktop/test.png",IMREAD_UNCHANGED);
cout<<"imread : "<<pngimg.channels() <<"\n";
imwrite("c:/Users/Laurent.PC-LAURENT-VISI/Desktop/test1.png",pngimg);
pngimg = imread("c:/Users/Laurent.PC-LAURENT-VISI/Desktop/test1.png", IMREAD_UNCHANGED);
cout << "imread after imwrite: " << pngimg.channels() << "\n";

result :

imread : 4
imread after imwrite: 4

On my disk image saved is good. Imread is called decoder->readData( *data ); exactly as like imdecode. may be I'm wrong but I don't think problem is in opencv

LBerger gravatar imageLBerger ( 2016-12-20 14:32:26 -0600 )edit

Than you for your time and for your answer. I performed a similar test and the result agree with your test. I still wonder if there is something wrong in my c++ code in the way that I construct data_mat_medium from buffer and in the decoding process. Also I think it's unfair for the question to be downvoted for mentioning the C# side of the code. I present the c# side to show that just before sending the image, this image really had an alpha channel and I think that this is important. I can remove the unity and c# stuff from the question if it is better for everyone. I apologize again and thank you one more time.

Nikolaos gravatar imageNikolaos ( 2016-12-20 23:34:53 -0600 )edit

Opencv is a large lib 20000 commits for 8 years.May be 30 tags since 2.2 version and last one is 3.2.0-rc. I have tested with 3.2.0-rc. If you tested too and found same result problem is not in opencv.

There is no problem to help you for your problem if you can give a piece code in c++ java or python and give OS and opencv version.

LBerger gravatar imageLBerger ( 2016-12-21 02:50:39 -0600 )edit

I found a solution by replacing the lines

std::vector<uchar> vectordataImMedium(imMedium, imMedium + imMediumLength);

cv::Mat data_mat_medium(vectordataImMedium, true);

mediumImage = cv::imdecode(data_mat_medium, 1);

with these

cv::Mat imgBuf(cv::Size(1280, 960), CV_8UC4, imMedium);

mediumImage= cv::imdecode(imgBuf, CV_LOAD_IMAGE_UNCHANGED);

Thank you again for time and your interest.

Nikolaos gravatar imageNikolaos ( 2016-12-21 06:21:47 -0600 )edit

Can you try without imdecode ?

mediumImage= imgBuf;
LBerger gravatar imageLBerger ( 2016-12-21 06:32:12 -0600 )edit

I tried that but the third party application crashed. After debugging the DLL visual studio gave me this message

Unhandled exception at 0x00007ffa401dc58a (opencv_imgcodecs310d.dll) in Unity.exe: 0xC0000005: Access violation reading location 0x000000002476d000.

at

line 831 : memcpy(png_ptr->row_buf + 1, row, row_info.rowbytes);

Nikolaos gravatar imageNikolaos ( 2016-12-21 06:53:43 -0600 )edit

Try to do not use imdecode :

cv::Mat mediumImage(cv::Size(1280, 960), CV_8UC4, imMedium);
LBerger gravatar imageLBerger ( 2016-12-21 07:35:01 -0600 )edit

Sorry for the delay (too much paperwork), not using imdecode doesn't seem to work, here are 5 steps from the debugging process (it shows with imdecode - which works and without imdecode - which deosn't work with some info from the vs debugger) https://s30.postimg.org/j3hqd5601/Ste...https://s30.postimg.org/ndwe8qb35/Ste...https://s30.postimg.org/hr4mozxy9/Ste...https://s30.postimg.org/meaoqrlb5/Ste...https://s30.postimg.org/fp45aqzz5/Ste...

note at step4 the "prevAllocator=????" , it doesn't look right.

Nikolaos gravatar imageNikolaos ( 2016-12-22 13:37:34 -0600 )edit

Something is wrong in your code. Try this small sample :

fstream fs;
fs.open("f:\\lib\\opencv\\samples\\data\\opencv-logo-white.png",ios::binary|ios::in);
fs.seekp(0, ios_base::end);
int nbByte=fs.tellp();
fs.seekp(0,ios_base::beg);
vector<char> tmp(nbByte);
fs.read(tmp.data(),nbByte);// in buffer I have got a png image 
Mat img=imdecode(tmp,IMREAD_UNCHANGED); // image is decode now
cout<<img.rows<<"\t"<<img.cols<<"\n";

You example should work like my example

LBerger gravatar imageLBerger ( 2016-12-22 14:12:39 -0600 )edit