Ask Your Question

16bit raw file stream analysis

asked 2015-03-03 13:32:31 -0500

anatrong gravatar image


I would like to analyse binary stream data. Data is 16-bit binary stream of 256x256 grayscale images with 256 bytes gaps stored in one long stream file. This is effectively video. Can you give me any advice how such file can be loaded to to openCV? Also files might be very big (>8GB), so I would be grateful if someone can point me to a way how to access this file sequentially.

I can imagine, this can be done on video files, but I did not find any codec supporting 16bit grayscale and I would actually like to avoid any conversion. Though 16bit can be possibly remapped into 24bit rgb space, but this looks highly speculative.

I included example of 16bit 256x256 file with 2 images, there is 256 bytes heather and 256 bytes gap between them.

C:\fakepath\2images.jpg (this is not jpg, only raw file)

edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted

answered 2015-03-04 12:47:35 -0500

LBerger gravatar image

updated 2015-03-04 13:28:27 -0500

Hi I think that you have to write your own code to read your image. For the first image in your file you can use something like this

imAcq =new ImageInfoCV(256,256,CV_16UC1);
ifstream fs;
float t[10];,ios::binary);
unsigned short *tmp=new unsigned short[256*256];


for (int i=0;i<256;i++)
    for (int j=0;j<256;j++)
        imAcq->at< unsigned short >(i,j)=tmp[i*256+j];

delete tmp;

After you can save your image in yml format. Then it depends what you want to do with your image. Some functions in OpenCV accept 16 bits images and some does not. You have to check this in doc before starting coding.

A screen copy of first image in your raw file image description

and yml file

edit flag offensive delete link more


Instead of the 256x256 loop, it is probably faster to use

Mat imAcq=Mat(256,256,CV_16UC1,&tmp,256*2).clone()

The optional ".clone()" makes an independent copy of the image, allowing you to reuse tmp for the next image or to dispose of it without corrupting imAcq.

Guyygarty gravatar imageGuyygarty ( 2015-03-04 13:28:17 -0500 )edit

Hi, I got back to this and I am stuck bit further.. this code:

    int main( int argc, char** argv )
        ifstream fs;[1], ios::in | ios::binary);
        unsigned short *tmp = new unsigned short[256*256];
        fs.seekg(256);<char*>(tmp), 256*256*sizeof(unsigned short) );

    Mat imAcq=Mat(256,256,CV_16UC1,&tmp,256*256*sizeof(unsigned short));
    for (long i=0; i<30000; i++)
    cout << " "  << tmp[i]/256;

creates tmp array with everything 256 x bigger, then if it is read in Mat imAcq=Mat(... it gets to segmentation fault. I am pretty sure my file is 16bit (unsigned short) and if I display values of tmp as in example it gets it right, any suggestions for quick, non all array fix?

anatrong0 gravatar imageanatrong0 ( 2015-06-09 10:11:14 -0500 )edit

Size is wrong (see Mat::Mat(int rows, int cols, int type, void* data, size_t step=AUTO_STEP)

int main(int argc, char** argv)
    ifstream fs;"14254110395737044.txt", ios::in | ios::binary);
    unsigned short *tmp = new unsigned short[256 * 256];
    fs.seekg(256);<char*>(tmp), 256 * 256 * sizeof(unsigned short));

    Mat imAcq = Mat(256, 256, CV_16UC1,tmp);// , tmp, 256 * 256 * sizeof(unsigned short));
    imshow("tets", imAcq);

LBerger gravatar imageLBerger ( 2015-06-09 10:56:46 -0500 )edit

OK thanks, that one works fine now. Other problem is now endians. Value 1 is given as 256 in my Mat. Is there any simple workaround? (if I multiply whole array by 256 in imageJ it gives me 1 for same pixel)

anatrong0 gravatar imageanatrong0 ( 2015-06-09 11:24:08 -0500 )edit

I do not understand 0x0100 = 256 and you want 0x0001 if yes I think something like that could be good

unsigned char*c = (unsigned char*)tmp;
    for (int i = 0; i < 65536*2; i += 2)
        swap(c[i], c[i + 1]);
    Mat imAcq = Mat(256, 256, CV_16UC1,tmp);// , tmp, 256 * 256 * sizeof(unsigned short));

and if you don't want most significant byte

unsigned char*c = (unsigned char*)tmp;
unsigned char cc[65536];
for (int i = 0; i < 65536 * 2; i += 2)
    swap(c[i], c[i + 1]);
    cc[i / 2] = c[i];
Mat imAcq = Mat(256, 256, CV_8UC1,cc);// , tmp, 256 * 256 * sizeof(unsigned short));
LBerger gravatar imageLBerger ( 2015-06-09 11:40:53 -0500 )edit

this seems to work:

Mat imAcq=Mat(256,256,CV_16UC1,tmp);
Mat mat1;
mat1 = imAcq/256;
imAcq = (imAcq - (256*mat1))*256+mat1;
anatrong0 gravatar imageanatrong0 ( 2015-06-09 12:20:41 -0500 )edit

v =0xFEDC




that's only the swap!

LBerger gravatar imageLBerger ( 2015-06-09 13:09:40 -0500 )edit

ok, I am not programmer as is clear I think :) is your suggestion faster?

anatrong0 gravatar imageanatrong0 ( 2015-06-09 13:37:31 -0500 )edit

Much more faster it's only a memory swap no arithmetic operation. If your camera is 16 bits why all Most significant bits are 0?

LBerger gravatar imageLBerger ( 2015-06-09 13:41:20 -0500 )edit

thanks for help, camera itself has 12bit counter and this is saved as 16bit raw file. It depends on our experiment how much of this space is filled. Counts are <20 in my example file (waste of space), but we do not have direct access to drivers or FPGA card which saves datasets. I'm doing analysis of data in proprietary software but I want to try something different.

anatrong0 gravatar imageanatrong0 ( 2015-06-09 13:57:00 -0500 )edit

answered 2015-06-09 13:17:09 -0500

anatrong0 gravatar image

updated 2015-06-09 14:27:06 -0500

for anyone doing simmilar, to display normalised first image from 16bit 256x256 raw binary file with 256 bytes header, there is also endian conversion (many thanks to L.Berger)

#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <iostream>
#include <fstream>          

using namespace cv;
using namespace std;

int main( int argc, char** argv )
        //file read
        ifstream fs;[1], ios::in | ios::binary);
        unsigned short *tmp = new unsigned short [256*256];
        fs.seekg(256);*)tmp, 256*256*2 );

    //endian conversion
    unsigned char*c = (unsigned char*)tmp;
    for (int i = 0; i < 65536*2; i += 2)
    swap(c[i], c[i + 1]);
    Mat imAcq = Mat(256, 256, CV_16UC1,tmp);// , tmp, 256 * 256 * sizeof(unsigned short));

    normalize(imAcq, imAcq, 0, 65536, NORM_MINMAX, -1, Mat() );

    //convert to 8bit for imshow
    Mat img8bit;
    namedWindow( "8bit normalised", WINDOW_AUTOSIZE );
    imshow( "8bit normalised", img8bit );                   // Show image

    waitKey(0);                                          // Wait for a keystroke in the window
    return 0;
edit flag offensive delete link more

Question Tools



Asked: 2015-03-03 13:32:31 -0500

Seen: 3,570 times

Last updated: Jun 09 '15