Doug Kerr
Well-known member
Although we rarely speak of it, a set of raw data from a particular camera type represents image color using a specific color space, the native color space of the sensor itself. Here, I will discuss that a little. This discussion is in a sense excerpted from my forthcoming article, "The Color Space of a Digital Camera Sensor".
Color spaces
In our context a color space means a fully-defined system of representing color by the values of (usually) three coordinates.
Tristimulus color spaces
The familiar "RGB" family of color spaces can be described as nonlinear tristimulus color spaces. The three values that define a color, R, G, and B, are nonlinear forms of three underlying values, r, g, and b. Those values tell us the "amounts" of three kinds of physical light, called the primaries, which if presented together to a viewer in those amounts will exhibit the color of interest.
There are tristimulus color spaces, such as the CIE XYZ color space, which use the same principle, but with an important wrinkle: their three primaries are not physically-realizable kinds of light, or even of non-visible radiation. We cannot, even in a laboratory, construct a physical emission of the "X" primary.
Nevertheless, these imaginary "primaries" follow the laws governing the color resulting from the mixture of primaries, and so, on paper, numerous combinations of amounts of these three primaries will indeed correspond to colors of light.
(One is tempted to say "visible light", but of course, rigorously, only visible radiation is "light", and in any case only visible radiation has "color", which is a perceptual property.)
The color space of a digital camera sensor
The outputs of the sensels of a conventional digital camera describe color in a system of three coordinates - the values of the outputs of the three classes of sensel detectors.
This situation constitutes a color space - the "native color space" of the sensor. We can also think of it as the color space of the raw data, which (more-or-less verbatim) records the sensel outputs.
We may be thrown off the track here by the fact that, in a CFA sensor, we do not get three values for any given pixel. That, however, does not disrupt the existence of a three-coordinate color space.
Recall that, even with a true "tricolor" sensor, we do not observe the color of every point in the image (there are an infinite number of those). Rather, we "sample" it at discrete points, at the pixel pitch.
The entire image (as, say, presented in a paper print), is derived from interpolation of those discrete color determinations (in part by the perhaps-Gaussian distribution of the ink patterns).
In a CFA sensor, we sample the three coordinates of the image color on staggered grids of discrete points, but the concept is no different because of that.
A metaphor might be a survey project in which we take data for all the trees in a certain forest, but, for reasons of data economy, for each group of three trees, we record for one only its latitude, for the next only its longitude, and for the third only its altitude. This does not any way invalidate the notion that this data is taken under the standard latitude/longitude/altitude coordinate system.
If we wanted to determine the color space of a CFA sensor in a laboratory, we would presumably do so with targets of uniform color, so concern with this matter would disappear.
How does demosaicing get into this?
The color space of which I speak is that of the sensor itself, meaning observed at the point where the sensel outputs are developed - in digital form, if you wish. This is before demosaicing. So demosaicing is not involved. (It may get tangled up in some of the implications, of course.)
There are many demosaicing algorithms, and it may well be that the color space of the image as first demosaiced is a different color space - a transform of the sensor native color space.
The primaries of the sensor native color space
We can readily guess (correctly) that the digital sensor native color space is of the tristimulus type. And in fact (for comfort), we usually label the three coordinates "R", "G", and "B".
Does that mean that this tristimulus color space works with primaries that have chromaticities something like the chromaticities of the R, G, and B primaries in common RGB-family color spaces?
No. In fact, the primaries of the native sensor color space, for most cameras that I know of, are imaginary (just like our infamous friends X, Y, and Z). This is of necessity so; a demonstration of this will be found in my article (which I need to finish up and publish).
On a related point, from here on, to avoid any misunderstanding, I will refer to the outputs of the three classes of sensels (and the sensel classes themselves) as R, G, and B .
Suppose that (to get rid of the demosaicing worry) we consider the sensor as regarding an image of uniform color. We look at the outputs of the R, G, and B sensel detector groups; we will imagine that all R outputs are the same value, and so forth, ignoring spatial and temporal noise.
Could we now in a laboratory, where we can construct light emitters having any spectrum we wish, construct three emitters (our primary sources) such that if we "throttled" them in proportion to the R, G, and B outputs of the sensor they would, for any color regarded by the sensor, create for the viewer that color? No.
Nevertheless, a valid set of three coordinate values describes a color.
Transformation from the native color space
We often see data (in for example the DxO labs reports on specific cameras) tables of the "sensitivities" of the three sensel groups, and often a transformation matrix. (These are related, in an obscure way.)
The purpose of that matrix is to define a transformation of the R, G, and B values from the sensor into r, g, and b values for some particular RGB-family color space (perhaps sRGB). From those, by applying the defined nonlinear transform, the R, G, and B values of the color in that color space would be developed.
This transformation may, in whole or in part, be embedded into the demosaicing algorithm of the camera or of external raw development software.
There are many complications here, relating to the spectrum of light under which the sensel detector sensitivities are determined. These are beyond the scope of this note.
Best regards,
Doug
Color spaces
In our context a color space means a fully-defined system of representing color by the values of (usually) three coordinates.
Tristimulus color spaces
The familiar "RGB" family of color spaces can be described as nonlinear tristimulus color spaces. The three values that define a color, R, G, and B, are nonlinear forms of three underlying values, r, g, and b. Those values tell us the "amounts" of three kinds of physical light, called the primaries, which if presented together to a viewer in those amounts will exhibit the color of interest.
There are tristimulus color spaces, such as the CIE XYZ color space, which use the same principle, but with an important wrinkle: their three primaries are not physically-realizable kinds of light, or even of non-visible radiation. We cannot, even in a laboratory, construct a physical emission of the "X" primary.
Nevertheless, these imaginary "primaries" follow the laws governing the color resulting from the mixture of primaries, and so, on paper, numerous combinations of amounts of these three primaries will indeed correspond to colors of light.
(One is tempted to say "visible light", but of course, rigorously, only visible radiation is "light", and in any case only visible radiation has "color", which is a perceptual property.)
The color space of a digital camera sensor
The outputs of the sensels of a conventional digital camera describe color in a system of three coordinates - the values of the outputs of the three classes of sensel detectors.
This situation constitutes a color space - the "native color space" of the sensor. We can also think of it as the color space of the raw data, which (more-or-less verbatim) records the sensel outputs.
We may be thrown off the track here by the fact that, in a CFA sensor, we do not get three values for any given pixel. That, however, does not disrupt the existence of a three-coordinate color space.
Recall that, even with a true "tricolor" sensor, we do not observe the color of every point in the image (there are an infinite number of those). Rather, we "sample" it at discrete points, at the pixel pitch.
The entire image (as, say, presented in a paper print), is derived from interpolation of those discrete color determinations (in part by the perhaps-Gaussian distribution of the ink patterns).
In a CFA sensor, we sample the three coordinates of the image color on staggered grids of discrete points, but the concept is no different because of that.
A metaphor might be a survey project in which we take data for all the trees in a certain forest, but, for reasons of data economy, for each group of three trees, we record for one only its latitude, for the next only its longitude, and for the third only its altitude. This does not any way invalidate the notion that this data is taken under the standard latitude/longitude/altitude coordinate system.
If we wanted to determine the color space of a CFA sensor in a laboratory, we would presumably do so with targets of uniform color, so concern with this matter would disappear.
How does demosaicing get into this?
The color space of which I speak is that of the sensor itself, meaning observed at the point where the sensel outputs are developed - in digital form, if you wish. This is before demosaicing. So demosaicing is not involved. (It may get tangled up in some of the implications, of course.)
There are many demosaicing algorithms, and it may well be that the color space of the image as first demosaiced is a different color space - a transform of the sensor native color space.
The primaries of the sensor native color space
We can readily guess (correctly) that the digital sensor native color space is of the tristimulus type. And in fact (for comfort), we usually label the three coordinates "R", "G", and "B".
Does that mean that this tristimulus color space works with primaries that have chromaticities something like the chromaticities of the R, G, and B primaries in common RGB-family color spaces?
No. In fact, the primaries of the native sensor color space, for most cameras that I know of, are imaginary (just like our infamous friends X, Y, and Z). This is of necessity so; a demonstration of this will be found in my article (which I need to finish up and publish).
On a related point, from here on, to avoid any misunderstanding, I will refer to the outputs of the three classes of sensels (and the sensel classes themselves) as R, G, and B .
Suppose that (to get rid of the demosaicing worry) we consider the sensor as regarding an image of uniform color. We look at the outputs of the R, G, and B sensel detector groups; we will imagine that all R outputs are the same value, and so forth, ignoring spatial and temporal noise.
Could we now in a laboratory, where we can construct light emitters having any spectrum we wish, construct three emitters (our primary sources) such that if we "throttled" them in proportion to the R, G, and B outputs of the sensor they would, for any color regarded by the sensor, create for the viewer that color? No.
Nevertheless, a valid set of three coordinate values describes a color.
Transformation from the native color space
We often see data (in for example the DxO labs reports on specific cameras) tables of the "sensitivities" of the three sensel groups, and often a transformation matrix. (These are related, in an obscure way.)
The purpose of that matrix is to define a transformation of the R, G, and B values from the sensor into r, g, and b values for some particular RGB-family color space (perhaps sRGB). From those, by applying the defined nonlinear transform, the R, G, and B values of the color in that color space would be developed.
This transformation may, in whole or in part, be embedded into the demosaicing algorithm of the camera or of external raw development software.
There are many complications here, relating to the spectrum of light under which the sensel detector sensitivities are determined. These are beyond the scope of this note.
Best regards,
Doug