Ask Your Question

A very basic question about image data

asked 2017-04-08 08:37:29 -0500

infoclogged gravatar image

Complete newbie in Image processing:

Normally the cameras saves the pictures in a specific format like jpeg or png. Videos are also encoded in formats like dvi, mpeg etc and saved onto the SD card.

My question is that how are these images laid out? There has to be some kind of a layout plan so that the digital sensor values are displayed in a particular layout. So along with the intensity information, the correpsonding pixel location should also be stored as well. For example row = 10, column = 20 contains digital intensity values RGB = 0xABCDEF.

Lets say, the format is not png or jpeg, but RAW. How will the data look like then? Would you get any detailed information?

Is there anything lower than the RAW data, that can be delivered by the cameras. As far as I have read, even the RAW formatted data is filtered and is not the exact snapshot of what a real hardware sensor sees.

When the opencv function ( VideoCapture() ) procures a frame of a video ( for example a running web cam ), then is it the same as the RAW format image captured from any of the digital cameras or is it already filtered? Well basically, this question regarding the webcam depends on my understanding of previous questions. Hence as of now, I will put it on hold.

edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted

answered 2017-04-08 17:02:49 -0500

Tetragramm gravatar image

I believe that RAW files are (with a few exceptions) as close as you can get to the pixel readouts, and usually has no filtering or image altering processing. There are some scientific cameras that just dump straight to their output, but that's typically due to very high frame rates. There are also a few cameras that do some processing before saving a RAW, but wikipedia says that they are rare and disliked for doing so.

As to the layout of any particular format, you would have to check the format specifications. With the exception of a bitmap, there is almost always many things going on. Some formats are lossy, and others are lossless. PNG is lossless, so whatever information you save is exactly what you get out. JPEG is lossy, so what you read is not necessarily what you saved.

So you have a PNG or JPEG image. What does the data look like? Who cares? What matters is what you get after it's read. When you call the imread function, it decodes the image into a pixel buffer. This has rows and columns and in those rows and columns it stores the digital intensity values (usually as BGR not RGB). If you print the memory as values you'll see 0xABCDEF. So that's good.

But, that is not what is in the RAW format for a given camera. Often RAW images store more bytes of data, say 12 bits per color per pixel. So the R would be 0x0000 to 0x1FFF, and the same for the B and G. If you have something that can read your raw format, it likely gets decoded into something like that. Except if it's a Bayer camera, which it probably is, you don't have RGB pixels at the same place, they're in a grid. So it's not useable directly. You have to process it before it can be used.

Typically after all the processing is done, you have compressed it down to an 8-bit image 0x00 to 0xFF with RGB for every pixel and then you save it as a PNG or JPEG. That is what is usable by a human, and what can be displayed on a monitor. A few monitors allow 10 or even 12 bits of display range, but they are typically for professionals only.

Webcams rarely offer the ability to get RAW images, and provided only the filtered and compressed images. VideoCapture then decodes them into the pixel buffer for you to use.

To get RAW video live, you typically need a scientific camera that outputs in Coax-Express, Camera Link, Gig-E, or now there are a few USB Vision.

edit flag offensive delete link more


The point where you said "When you call the imread function ...", my question is how does the imread function map the pixel value to the row and column? There has to be a layout and this is what I want to know, how does the layout of a RAW image in the memory looks like? Using metadata? I care how the data is stored on the computer and not how the data is read from the computer. Regarding webcams, do you mean, the internals of the hardware in the webcam has reduced the size of the image in order to be fast enough to offer the frames to the computer motherboard? Or what might be the reason, that the webcams, do not offer RAW images? Cost? I would be interested to know, what would have been the approximate data volume in RAW format when shipping the data from the webcam ...(more)

infoclogged gravatar imageinfoclogged ( 2017-04-09 01:42:25 -0500 )edit

It's complicated imread uses libjpeg and libpng. How the files actually store the data, I have very little idea, and I don't particularly care. Here are the wiki pages for PNG and JPEG files. Both are complicated.

As for how RAW images work? There are apparently at least 42 different ways, based on counting the formats on the wiki page, and I bet some of those file extensions have more than one internal format they use.

Since there's no standard (or at least any that are used by anyone...), a webcam can't provide a standard interface to get RAW images. Since RAW images have to be post-processed to be used, anyone who wants RAW will just use a real camera, so why bother?

Tetragramm gravatar imageTetragramm ( 2017-04-09 16:27:27 -0500 )edit

As for data volume, do the math yourself. actual readout pixels * frame rate * bit depth = rate in bits/second. Then go compare that to USB 2.0 speeds of 280e6 bit/s and you'll see why.

Tetragramm gravatar imageTetragramm ( 2017-04-09 16:30:27 -0500 )edit

answered 2017-04-08 15:53:28 -0500

dohnjoe gravatar image

I dont quite understand what you're trying to ask here but I hope you do realize that PNG, JPEG and RAW refer to the image compression methods (basically a turn on how you store images) and RGB values refer to the colorspace in which images are saved. An image could be saved in any format and it need not be RGB all the time (well but it technically is due to RGB being a common standard).

If you're asking about the differences between PNG, JPEG, and RAW. Then I would suggest reading up on image compression

When VideoCapture procures a frame from a video of a file then the answer to your question would be "what is format of the video file?" moreover "what is the compression method used for the video file?" I would suggest reading up on video compression formats / video codecs.

I would suggest framing the question with a clear understanding as to what it is you want to ask about would always be a good start.

edit flag offensive delete link more


why dont you grep for question marks ? ;) by the way, RAW is not in the samel liga as JPEG and PNG. It does not refer to the image compression methods.

infoclogged gravatar imageinfoclogged ( 2017-04-09 01:48:06 -0500 )edit

Look refer to tetragramm's answer below, he puts it plainly. RAW is when do ZERO compression to the data before you save it (usually). You save it as it as. JPEG and PNG are compression methods in which you selectively choose certain kinds of values and throw the rest of them away, which you would have known, had you looked up image compression methods as I had asked you to.

I hope you do realize the contradiction in your own statement. You yourself brought in jpeg and raw into the mix when there was no need to

"specific format like jpeg or png".

I answered in the context in which you asked the question in, if you want an answer to something else, you should ask accordingly.

As I said, next time please frame the question with a specific goal of what you actually want.

dohnjoe gravatar imagedohnjoe ( 2017-04-09 13:51:29 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower


Asked: 2017-04-08 08:37:29 -0500

Seen: 913 times

Last updated: Apr 08 '17