Hi All,
I've seen some similar questions asked around here and on StackOverflow (and I've even posted there), but here's my trouble: I've got a .cr2 image shot on a Canon Rebel T5 (with the camera set to take RAW sRGB images), and when I load and De-Bayer the image using LibRaw and OpenCV3 my image seems a bit too yellow and bright.
Here's what I expect
And here's what I get (the different dimensions are due to bad crop, it's the color I'm talking about)
I started by using LibRaw to open and unpack the raw image file.
// Open the CR2 file with LibRaw, unpack, and create image
LibRaw lrProc;
assert( LIBRAW_SUCCESS == lrProc.open_file( "001.cr2" ) );
assert( LIBRAW_SUCCESS == lrProc.unpack() );
assert( LIBRAW_SUCCESS == lrProc.raw2image() );
This example on LibRaw's website indicates that I'll be left with an "image" where each pixel is actual 4 ushort values [Red, Green1, Blue, Green2] representing the color at that pixel of the bayered image. When I print out the values (using the following code)
// Print the first 4 values of the first 4 rows
for ( int y = 0; y < 4; y++ )
{
for ( int x = 0; x < 4; x++ )
{
int idx = y * width + x;
ushort * uRGBG = lrProc.imgdata.image[idx];
printf( "[%04d, %04d, %04d, %04d] ", uRGBG[0], uRGBG[1], uRGBG[2], uRGBG[3] );
}
printf( "\n" );
}
I get the following results:
[2253, 0000, 0000, 0000] [0000, 2166, 0000, 0000] [2183, 0000, 0000, 0000] [0000, 2195, 0000, 0000]
[0000, 0000, 0000, 2207] [0000, 0000, 2175, 0000] [0000, 0000, 0000, 2099] [0000, 0000, 2122, 0000]
[2246, 0000, 0000, 0000] [0000, 2240, 0000, 0000] [2287, 0000, 0000, 0000] [0000, 2182, 0000, 0000]
[0000, 0000, 0000, 2251] [0000, 0000, 2103, 0000] [0000, 0000, 0000, 2195] [0000, 0000, 2155, 0000]
So on even rows the Red and Green1 pixels have alternatively nonzero values, and on odd rows the Blue and Green2 pixels have alternatively nozero values. The values seem to be 12 bit, although I'm not 100% on that.
Looking at the description of OpenCV's cvtColor function as well as this person's description of some of the cr2 color formats, indicates to me that I've got a Bayered image of the "BG" variety (2nd row 2nd column is blue, 2nd row 3rd column is green.)
My assumption was that in order to debayer the image using cvtColor I have to take the bayer pixel values and copy them into a contiguous Mat buffer for cvtColor
// Get image dimensions
int width = lrProc.imgdata.sizes.iwidth;
int height = lrProc.imgdata.sizes.iheight;
// Create a buffer of ushorts containing the pixel values of the "BG Bayered" image
std::vector<ushort> vBayerData;
for ( int y = 0; y < height; y++ )
{
for ( int x = 0; x < width; x++ )
{
// Get pixel idx
int idx = y * width + x;
// Each pixel is an array of 4 shorts rgbg
ushort * uRGBG = lrProc.imgdata.image[idx];
// For even rows, get either red or green, store in vec
if ( y % 2 == 0 )
{
bool red = x % 2 == 0;
vBayerData.push_back( uRGBG[red ? 0 : 1] );
}
// For odd rows, get either blue or green
else
{
bool green = x % 2 == 0;
vBayerData.push_back( uRGBG[green ? 3 : 2] );
}
}
}
// Construct openCV mat
cv::Mat imgBayer( height, width, CV_16UC1, vBayerData.data() );
// Debayer image, get output
cv::Mat imgDeBayer;
cv::cvtColor( imgBayer, imgDeBayer, CV_BayerBG2BGR );
if I examine imgDeBayer I see that it's very dark. However, because I have 12 bit pixel values in a 16 bit space, I've got to bit shift my pixel values by 4 to get them in the correct range (which I believe is equivalent to a multiplication by 16
// The pixel color values were 12 bit, but our data is 16 bit
// transform the range [0, 4095] to [0:65535] (multiply by 16)
imgDeBayer *= 16;
However, what I'm left with is the image at the top of this post, which seems off color. I'm wondering if it's a gamma correction issue (seeing as the camera settings says sRGB), but I've experimented with a few different values and not had any luck (using the code below, which may be invalid)
// Remap to float [0, 1], perform gamma correction
float fGamma = 2.2f;
imgDeBayer.convertTo( imgDeBayer, CV_32FC3, 1. / USHRT_MAX );
cv::pow( imgDeBayer, fGamma, imgDeBayer );
I'm not sure if the gamma correction step is unnecessary, but setting the value to 1 (i.e skipping it) doesn't make the output correct. I've also tried different bayer conversion enums (i.e BayerRG) to no avail. My suspicion is that I'm copying color component values from the bayered image incorrectly, but changing that up seems to yield worse results.
Does anyone have any suggestions? Or is anyone able to recognize what's causing the problem in my output image's color / brightness?
Thanks for reading, and let me know if I can further clarify. Here is my code in it's entirety
// Open the CR2 file with LibRaw, unpack, and create image
LibRaw lrProc;
assert( LIBRAW_SUCCESS == lrProc.open_file( "001.cr2" ) );
assert( LIBRAW_SUCCESS == lrProc.unpack() );
assert( LIBRAW_SUCCESS == lrProc.raw2image() );
// Get image dimensions
int width = lrProc.imgdata.sizes.iwidth;
int height = lrProc.imgdata.sizes.iheight;
// Create a buffer of ushorts containing the pixel values of the "BG Bayered" image
std::vector<ushort> vBayerData;
for ( int y = 0; y < height; y++ )
{
for ( int x = 0; x < width; x++ )
{
// Get pixel idx
int idx = y * width + x;
// Each pixel is an array of 4 shorts rgbg
ushort * uRGBG = lrProc.imgdata.image[idx];
// Even rows are RGRGRG..., odd are GBGBGB...
// For even rows, get either red or green, store in vec
if ( y % 2 == 0 )
{
bool red = x % 2 == 0;
vBayerData.push_back( uRGBG[red ? 0 : 1] );
}
// For odd rows, get either blue or green
else
{
bool green = x % 2 == 0;
vBayerData.push_back( uRGBG[green ? 3 : 2] );
}
}
}
// Get rid of libraw image, construct openCV mat
lrProc.recycle();
cv::Mat imgBayer( height, width, CV_16UC1, vBayerData.data() );
// Debayer image, get output
cv::Mat imgDeBayer;
cv::cvtColor( imgBayer, imgDeBayer, CV_BayerBG2BGR );
// The pixel color values were 12 bit, but our data is 16 bit
// transform the range [0, 4095] to [0:65535] (multiply by 16)
imgDeBayer *= 16;
// Remap to float [0, 1], perform gamma correction
float fGamma = 2.2f;
imgDeBayer.convertTo( imgDeBayer, CV_32FC3, 1. / USHRT_MAX );
cv::pow( imgDeBayer, fGamma, imgDeBayer );
// Display image
cv::namedWindow( "CR2 File", CV_WINDOW_FREERATIO );
cv::imshow( "CR2 File", imgDeBayer );
cv::waitKey();
-John