sRaw redux

When the new Canon "sRaw" image format emerged (on the EOS 1D Mark III), I paid little attention. I did note that there seemed to be some initial lack of clarity as to just exactly what this creature was. That uncertainty has been very durable.

Just today, I was reminded of the matter by an inquiry posted on the Pro Photo Home forum. I thought it was about time to figure out "what is really going on here".

You can follow the action over there by starting here:

http://www.prophotohome.com/forum/c...x-sensors/89799-what-sraw-camera-process.html

A couple of the wizards on the forum pointed me to a couple of "articles" on the matter, and I was off and running.

I'm a long way from figuring out the whole thing, but certain things seem likely so. Here's what I today believe is the story.

**********

• The sRaw format occupies an interesting middle ground between the familiar set of raw data and a "developed" image. It is "slightly-cooked" data - essentially demosaiced and that's all

• In the "basic" version, the image represented in sRaw form has exactly the pixel dimensions of one of the small options the camera offers for its JPEG output (in the EOS 40D, equivalent to the ""small" JPEG output).

• For each pixel position (at this resolution), there are an R, G, and B value, in linear, 14-bit form, which can be thought of as demosaiced and "downsampled" raw data (and with the two "G" channels consolidated; since the data has already been demosaiced, there is no value nor meaning to two G values per pixel).

• These are then transformed to a YCbCr form, with the Cb and Cr components at half the resolution of the "sRaw fat pixels" (just as is done, in our regular JPEG file, with the gamma-precompensated RGB values of the pixels of the developed image). Just as in the JPEG case, this recognizes the lower resolution of the human eye to changes in chromaticity compared to luminance. In effect, the raw data has been compromised in this regard.

• The resulting suite of data comprises four 14-bit numbers for each two "sRaw pixels" (2 Y values, 1 Cb value, and 1 Cr value).

• This is then compressed with a "reversible" JPEG compression algorithm ("lossless" is the unfortunate common description).

Now, with such an sRaw file in hand, we can decode the compression (since the compression is reversible, we get the original data back precisely) and, by interpolating the Cb and Cr values (since there is only one of each for every two sRaw pixels), get a complete YCbCr representation for each sRaw pixel.

From that, we can unambiguously reconstruct the (linear) R, G, and B "raw" values for each sRaw pixel.

We now work this just as we would work "regular" raw data - except that we do not need to demosaic it, since we already have an explicit set of R, G, and B for each sRaw pixel. We can manage it for "exposure compensation", apply a color-correction vector for white balance purposes, and apply sharpening.

**********

If the sRaw image has half the resolution (half the pixel dimensions) of the "native" resolution of the camera, then the amount of data to be compressed is exactly half the amount of data for the regular raw file (of the 14-bit flavor).

**********

What is the object of this, anyway? Canon, in the white paper for the EOS 1D Mark III, where this format first appeared, points out that the sRaw file (of the single type then available) has about half the size of a conventional raw file, and supports an image of half the pixel dimensions (1/4 the number of pixels).

They continue that such an image may be very important for some work (wedding photography is, inexplicably, mentioned), and it is advantageous to have the flexibility in processing offered by the raw approach with a smaller file than if the regular raw output were used.

Later (even in the white paper for the EOS 40D), they gave up trying to explain its advantage. And wedding photographers were not further mentioned.

**********

That's how it looks here. Others may know things that show this outlook to be a complete misconception.

Thanks Doug!

I've never used or thought of trying sRAW as if it was sired by the father of sRGB, which of course is just a hateful rumor.

My two eyes are looking different ways at this time after editing too many pictures and partaking of just enough fine wine to be agreeably relaxed. So I'll print out the article and comment tomorrow; if I can remember....

Asher

As I asked elsewhere when this came up, what possible benefit does sRaw provide? And does anyone else thing putting raw in there is a tad misleading?

Hi, Andrew

Andrew Rodney said:

As I asked elsewhere when this came up, what possible benefit does sRaw provide?

Click to expand...

Well, as I quoted above, Canon's answer to that (not heard so often anymore) goes like this (as I paraphrase it):

And does anyone else thing [think] putting raw in there is a tad misleading?

Click to expand...

I assume you are speaking of basing its name on the string "raw". Let me characterize the critter, so people can make up their own minds about that.

The sRaw file carries a "downsampled" set of the sensor output data (matching the resolution of the smaller image), with these further properties:

• It is already "demosaiced" (that is, the data is for "deduced" pixels of the reduced resolution image, not CFA sensels).

• The "luminance" and "chrominance" aspects are separated in the presentation, and the chrominance aspect is presented at a horizontal resolution half the image pixel resolution.

This still allows us to apply compensation for exposure "misadventures" and to apply white balance color correction, as we can with the conventional raw data.

It does not allow us to use what we might think is a superior demoscaicing algorithm, as we could with conventional raw data.


It does not fully preserve the chrominance detail implied by the downsampled sensor output data. (This is of course also true in all "standard" JPEG files.)

Best regards,

Doug

Doug_Kerr said:

• It is already "demosaiced"

Click to expand...

In my book, it ain’t raw then.

I've never done an analysis of sRAW, but when it first emerged it seemed to me that the advantage of a smaller 'raw' format could be based on some form of hardware pixel binning/combination to achieve improved noise results at high iso's. Since the sraw resolution does not allow this for all channels (if you preserve r and b spacial resolution) I discarded it as not worthwhile from my perspective.

Quite why a wedding photographer would remove their ability to make big expensive prints is quite beyond me. If your making a profit then buying a few (reuseable!) cards is no longer a big deal.

Mike

"Gao Gao" (I think that is actually Zang Gao), writing on dpr, presents an description of the sRaw data transformation and presentation. It largely comes from his reverse engineering of the source code in current versions of dcraw, an open-source program that "decodes" the raw file from many cameras and delivers an image in TIFF form (among other options). This is developed by Dave Coffin (the "dc" of "dcraw").

Coffin has in fact reverse-engineered the various algorithms involved, the camera manufacturers in general not disclosing the structure of their raw files.

The form of Gao's result I worked from indicates the "decoding" of the sRaw data (after "decoding" of its JPEG compression. I "inverted" his equations to get a description of the "encoding" of the sRaw data.

My result follows.

I will use Gao's notation for the three quantities in which the sRaw data is represented, C0, C1, and C2. These are conceptually parallel to the quantities Y, Cb, and Cr found in the YCbCr family of color spaces (including sYCC). But those have different specific definitions from what we find in sRaw (and in fact, the underlying quantities are different), so Gao uses different notation (a good idea, which I follow).

C0 can be considered to give the quasi-luminance implied by the downsampled sensel outputs at a pixel of the reduced-resolution image implied by an sRaw file. Unlike Y, it is not based on non-linear forms of R, B, and B. (But it does not follow the standard definition of luminance for any familiar set of primaries - thus my term "quasi-luminance".)

C1 and C2 (like Cb and Cr) describe the chrominance implied by the downsampled sensel outputs at a pixel of the reduced-resolution image implied by an sRaw file. Unlike Cb and Cr, they are not based on non-linear forms of R, B, and B.

C1 and C2 are only developed and included in the file for even-numbered pixels ("chrominance subsampling").

When reconstructing the downsampled raw data from an sRaw file (so the "development" of the image can be completed), C1 and C2 are derived for the odd-numbered pixels by interpolation between the C1 and C2 for the proceeding pixel and for the succeeding pixel.

The development of C0, C1, and C2 are given below.

The notation r, g, and b refers to the downsampled demosaiced sensor outputs (there are not two "g" values; there would be no point once demoscaicing has been done). I think these are not subject to any "black offset". All values here are in 14-bit form.

For each pixel:
C0= 0.112r + 0.592g + 0.296b

For even-numbered pixels only
C1=B-C0
C2=R-C0

Probably.

Best regards,

Doug

Hi, Mike,

Mike Shimwell said:

I've never done an analysis of sRAW, but when it first emerged it seemed to me that the advantage of a smaller 'raw' format could be based on some form of hardware pixel binning/combination to achieve improved noise results at high iso's. Since the sraw resolution does not allow this for all channels (if you preserve r and b spacial resolution) I discarded it as not worthwhile from my perspective.

Click to expand...

Putting aside the matter of chrominance downsampling, I think that "binning" advantages in going from native resolution to about a 1/4 pixel count image would be hard to come by, given that we have a CFA sensor system, not a "tricolor per pixel" sensor (as with the Foveon).

Looking at it in a primitive way, it takes four sensels to get a genuine pixel color (which we of course don't get in the full-size developed outputs)

So if in a reduced-size image we take advantage of the smaller pixel count to get a "genuine" pixel color, we probably can't get a binning advantage as well. "There ain't no free lunch."

Quite why a wedding photographer would remove their ability to make big expensive prints is quite beyond me. If your making a profit then buying a few (reuseable!) cards is no longer a big deal.

Click to expand...

Will Thompson just mentioned to me that this may be a notion based on the film practice of wedding photographers shooting the main shots in MF but using ff 35 mm for the "candids" (to save film and processing cost).

Best regards,

Doug

My reports so far have been on the "small" sRaw, originally the only kind there was.

In later cameras, a not-so-small ("medium") version appeared. At first it was called sRaw1, with the original ("small") one called in such cameras sRaw2 (go figger).

But later yet, the "medium" one is called mRaw and the original ("small") one sRaw again.

mRaw differs from sRaw thus:

• Its image size corresponds to the second-smallest JPEG image. Its total pixel count is roughly 1/2 that of the native image. (for sRaw, its pixel count is generally exactly 1/4 that of the native format.)

• Its chrominance is subsampled at 1:4 (for sRaw, it is 1:2); that is, there is only one set of C1 and C2 for every four pixels. (Thus, for four image pixels, there are four C0 values and one each of C1 and C2.) To use the peculiar formal notation for chrominance subsampling, sRaw is 4:2:2. mRaw is 4:2:0. Accordingly, the amount of data to be compressed is about 3/4 the amount in a raw file, about 1.5 times the amount in an sRaw file.

Best regards,

Doug

Hi Doug,

An excellent research and redux, thanks for doing all this hard work and keeping us informed! Much appreciated

Cheers,

I never saw the point in sRAW and still don't. Storage is cheap.

The pre-demosaicing precludes any future application of newer and better demosaicing algorithms, one of the reasons for shooting RAW in the first place.

Hi, Peter,

Peter Galbavy said:

I never saw the point in sRAW and still don't. Storage is cheap.

Click to expand...

Indeed, I suspect that sRaw is a solution that made it to the surface as the problem was about to disappear. There has been little horn-blowing about it from Canon since shortly after its introduction.

The pre-demosaicing precludes any future application of newer and better demosaicing algorithms, one of the reasons for shooting RAW in the first place.

Click to expand...

So it might seem (and so I thought until about 3:30 this morning).

However, when we take a suite of sensel outputs and seek to derive from it a "full-color" image at half the resolution (as we essentially do in the sRaw format), we can take the view that we are really not demosaicing, but rather operating from a true tristimulus pixel sensor, 2x2 sensel pitches in size (that is, each pixel of the image is picked by by one R, two G, and one B sensels dedicated to that pixel and that pixel alone). This is then very nearly equivalent to what we have in a Foveon sensor, or in a "three chip CCD" video camera, in which demosaicing as such does not take place.

So the concept of a "better" demosaicing algorithm in that case seems moot.

I do not mean to suggest that this is how the "development" of the half-resolution image represented by the sRaw data set is done. Perhaps something more tricky is done for reasons I am not at the moment prepared to recognize. And then of course that would leave open the concept that we could have done something more clever yet. And of course we still can by working from the Raw output.

(This story doesn't apply to the "mRaw" form, where clearly some form of demosaicing is required - the pixels there do not embrace a complete set of R,G1,G2,B sensels)

Curiously,when I took a pause to look at the forum traffic and saw your message, I was writing that very part of a new article I am preparing on the sRaw format (before I forget what I have figgered out).

Thanks for your observations.

Best regards,

Doug

Doug_Kerr said:

However, when we take a suite of sensel outputs and seek to derive from it a "full-color" image at half the resolution (as we essentially do in the sRaw format), we can take the view that we are really not demosaicing, but rather operating from a true tristimulus pixel sensor, 2x2 sensel pitches in size (that is, each pixel of the image is picked by by one R, two G, and one B sensels dedicated to that pixel and that pixel alone). This is then very nearly equivalent to what we have in a Foveon sensor, or in a "three chip CCD" video camera, in which demosaicing as such does not take place.

Click to expand...

... but you only get half the resolution and we cannot be sure about how the microlenses over the sensor sites spread the light. While the half-resolution would normally appear to defeat the problem by combining the light by a process of sharpening there are still cases where light may intend to fall only on one of the G sensels and not the other. Averaging is a very crude sharpening tool

Hi, Peter,

Peter Galbavy said:

... but you only get half the resolution and we cannot be sure about how the microlenses over the sensor sites spread the light. While the half-resolution would normally appear to defeat the problem by combining the light by a process of sharpening there are still cases where light may intend to fall only on one of the G sensels and not the other. Averaging is a very crude sharpening tool

Click to expand...

I understand.

Best regards,

Doug