De-Bayering a cr2 image?

asked 2016-10-23 12:43:30 -0500

mynameisjohnj gravatar image

Hi All,

I've seen some similar questions asked around here and on StackOverflow (and I've even posted there), but here's my trouble: I've got a .cr2 image shot on a Canon Rebel T5 (with the camera set to take RAW sRGB images), and when I load and De-Bayer the image using LibRaw and OpenCV3 my image seems a bit too yellow and bright.

Here's what I expect

The correct image

And here's what I get (the different dimensions are due to bad crop, it's the color I'm talking about)

My output

I started by using LibRaw to open and unpack the raw image file.

// Open the CR2 file with LibRaw, unpack, and create image
LibRaw lrProc;
assert( LIBRAW_SUCCESS == lrProc.open_file( "001.cr2" ) );
assert( LIBRAW_SUCCESS == lrProc.unpack() );
assert( LIBRAW_SUCCESS == lrProc.raw2image() );

This example on LibRaw's website indicates that I'll be left with an "image" where each pixel is actual 4 ushort values [Red, Green1, Blue, Green2] representing the color at that pixel of the bayered image. When I print out the values (using the following code)

// Print the first 4 values of the first 4 rows
for ( int y = 0; y < 4; y++ )
{
    for ( int x = 0; x < 4; x++ )
    {
        int idx = y * width + x;
        ushort * uRGBG = lrProc.imgdata.image[idx];
        printf( "[%04d, %04d, %04d, %04d]  ", uRGBG[0], uRGBG[1], uRGBG[2], uRGBG[3] );
    }
    printf( "\n" );
}

I get the following results:

[2253, 0000, 0000, 0000] [0000, 2166, 0000, 0000] [2183, 0000, 0000, 0000] [0000, 2195, 0000, 0000]
[0000, 0000, 0000, 2207] [0000, 0000, 2175, 0000] [0000, 0000, 0000, 2099] [0000, 0000, 2122, 0000]
[2246, 0000, 0000, 0000] [0000, 2240, 0000, 0000] [2287, 0000, 0000, 0000] [0000, 2182, 0000, 0000]
[0000, 0000, 0000, 2251] [0000, 0000, 2103, 0000] [0000, 0000, 0000, 2195] [0000, 0000, 2155, 0000]

So on even rows the Red and Green1 pixels have alternatively nonzero values, and on odd rows the Blue and Green2 pixels have alternatively nozero values. The values seem to be 12 bit, although I'm not 100% on that.

Looking at the description of OpenCV's cvtColor function as well as this person's description of some of the cr2 color formats, indicates to me that I've got a Bayered image of the "BG" variety (2nd row 2nd column is blue, 2nd row 3rd column is green.)

My assumption was that in order to debayer the image using cvtColor I have to take the bayer pixel values and copy them into a contiguous Mat buffer for cvtColor

// Get image dimensions
int width = lrProc.imgdata.sizes.iwidth;
int height = lrProc.imgdata.sizes.iheight;

// Create a buffer of ushorts containing the pixel values of the "BG Bayered" image
std::vector<ushort> vBayerData;
for ( int y = 0; y < height; y++ )
{
    for ( int x = 0; x < width; x++ )
    {
        // Get pixel idx
        int idx = y * width + x;

        // Each pixel is an array of 4 shorts rgbg
        ushort * uRGBG = lrProc.imgdata.image[idx];

        // For even rows, get either red or green, store in vec
        if ...
(more)
edit retag flag offensive close merge delete

Comments

Hi pls tell me, How did you obtained your destination(Expected output) Image? Is that from a Camera manufacturer SDK? If yes, you may need to look at the Generic Camera Pipeline! Your yellow color is may be due to lack of White balance!

Balaji R gravatar imageBalaji R ( 2016-10-23 22:39:21 -0500 )edit

Also these lines are unnecessary! // The pixel color values were 12 bit, but our data is 16 bit // transform the range [0, 4095] to [0:65535] (multiply by 16) imgDeBayer *= 16;

// Remap to float [0, 1], perform gamma correction
float fGamma = 2.2f;
imgDeBayer.convertTo( imgDeBayer, CV_32FC3, 1. / USHRT_MAX );
cv::pow( imgDeBayer, fGamma, imgDeBayer );

You can directly convert into 32-Bit Floating point Mat!

// Remap to float [0, 1], perform gamma correction
float fGamma = 2.2f;
imgDeBayer.convertTo( imgDeBayer, CV_32FC3, 1. / 4096);//4096-> 2^12 ( 12-Bit)
cv::pow( imgDeBayer, fGamma, imgDeBayer );
Balaji R gravatar imageBalaji R ( 2016-10-23 22:42:06 -0500 )edit

Hi Balaji, good point about that scale factor. A few of these steps can certainly be collapsed.

I got the "expected output" by just double clicking the image in Windows (it opens in the default "Photos" program), but I'm not aware of this pipeline you speak of so that's definitely a good lead! No powerpoint here unfortunately, but I'll find a way of opening that presentation. Thanks!

edit: here are some links on the pipeline https://pdfs.semanticscholar.org/519e...http://www.cs.cmu.edu/afs/cs/academic...

if anyone finds this. I'll return with any positive results.

mynameisjohnj gravatar imagemynameisjohnj ( 2016-10-23 22:59:42 -0500 )edit