CFA sensors and their Nyquist frequencies

[Doug Kerr]

We sometimes hear it said (in fact, I said it just this morning) that:

In a Bayer CFA array, the Nyquist frequency (and thus we might imagine the resolution) of the G layer is the same as for a monochrome array with the same sensel pitch.​

How can this be? The G layer has only half as many sensels as the monochrome sensor.

The key to this conundrum is that the statement above is true in part but not completely stated.

The issue is that the Nyquist frequency of a sensor (and thus its potential resolution) is not the same for detail along all axes.

A more complete statement would be:

In a conventionally-oriented Bayer CFA array, the Nyquist frequency (and thus we might imagine the resolution) of the G layer, for detail running in along the horizontal or vertical axis, is the same as for detail running in that same direction in a conventionally-oriented monochrome array with the same sensel pitch.​

To help us follow that notion, this figure represents a monochrome sensor (conventional orientation) with pixel pitch p.

Monochrome array​

In the left-hand panel we consider detail running along the horizontal or vertical axes. For the horizontal detail, the pitch of the pixels is that indicated by the vertical lines. They are at a spacing p, and so for the horizontal detail, the Nyquist frequency, FN, is 1/(2p) (and the resolution might be about 0.75 times that).

For the vertical detail, the pitch of the pixels is that indicated by the horizontal lines. They are at a spacing p, and so for the vertical detail, the Nyquist frequency, FN, is again 1/(2p) (and the resolution might be about 0.75 times that).

In the right-hand panel we consider detail running along an axis running upward and to the right at an angle of 45° to the horizontal. (The situation is comparable for the opposite diagonal axis, but I don't show that to avoid cluttering the figure.) For that detail, the pitch of the pixels is that indicated by the diagonal lines. They are at a spacing 0.707p, and so for the that diagonal detail, the Nyquist frequency, FN', is is 1.414 times the "mopnochrome" Nyquist frequency, FN, (and the resolution might be about 0.75 times that).

Wow! We have greater resolution potential for detail running along diagonal axes than for detail running along a vertical or horizontal axis. Can we take advantage of that? Sure. By shooting objects where the important detail runs along diagonal axes. Like what? A brick wall shot with the camera rolled at a 45° angle.

In fact, in one series of Fujifilm cameras, the sensor was in fact laid out at 45° to the conventional layout to exploit that fact.​

But back to our story. This figure represents the G layer of a Bayer CFA sensor (conventional orientation) with overall pixel pitch p.

Bayer CFA array - G layer​

As before, in the left-hand panel we consider detail running along the horizontal or vertical axes. For the horizontal detail, the pitch of the pixels is that indicated by the vertical lines. They are at a spacing p, and so for the horizontal detail, the Nyquist frequency, FN', is the same as the monochrome Nyquist frequency, FN (and again the resolution might be about 0.75 times that).

Of course the situation for vertical detail is comparable.

Thus, simplistically, for vertical or horizontal detail, there is no price paid in resultion for the lower number of sensels. (There are some subtle issues that make that not quite true, but they are beyond this discussion.)

Now in the right-hand panel, we consider detail running along the diagonal axis running upward and to the right. For that detail, the pitch of the pixels is that indicated by the diagonal lines. They are at a spacing 1.414p, and so for that diagonal detail, the Nyquist frequency, FN', is is 0.707 times the "monochrome" Nyquist frequency, FN, (and the resolution might be about 0.75 times that).

So here we have a lesser resolution for diagonal detail (the price paid for a lesser number of sensels that in the monochrome case).

This figure represents the R layer of a Bayer CFA sensor (conventional orientation) with overall pixel pitch p. Here, we have only 1/4 as many sensels as for the monochrome sensor.

Bayer CFA array - R layer​

By now you know the drill. In the left-hand panel we see that for the R layer, for horizontal or vertical detail, the Nyquist frequency is half that of the monochrome sensor.

In the right-hand panel we see the situation for diagonal detail. We see that here the applicable Nyquist frequency, FN', is 0.707 the Nyquist frequency for the monochrome sensor (FN).

This is greater than the Nyquist frequency for vertical and horizontal detail (as is the case with the monochrome sensor).

The situation for the B layer is identical.

Best regards,

Doug

 

[Asher Kelman]

Well, that's interesting, Doug!

As a consequence, would features on a curve that coincided with a diagonal have inherently increased resolution? So this would help in defining the pupil, nose, lips and other facial and body features. As long as the appearance was slightly sharper, the brain likely s not interprets the entire form as sharp.

I wonder if that's true?

Asher

 

[Doug Kerr]

Hi, Asher,

Well, that's interesting, Doug!

As a consequence, would features on a curve that coincided with a diagonal have inherently increased resolution?

Well, it would seem as if it might.

So this would help in defining the pupil, nose, lips and other facial and body features. As long as the appearance was slightly sharper, the brain likely s not interprets the entire form as sharp.

I wonder if that's true?

Well, there are a lot of interesting issues here.

An interesting parallel here is the matter of the resolution of a TV image, which in many cases is not the same in the vertical and horizontal directions (although nominally the allocation of bandwidth is intended to make them comparable).

As a result, for example, the ability to read fine print (a typed ransom note, for example) may vary with the orientation of the note (a matter that skilled cinematographers were known to keep in mind while the director was blocking scenes).

Best regards,

Doug

 

[Bart_van_der_Wolf]

As a consequence, would features on a curve that coincided with a diagonal have inherently increased resolution? So this would help in defining the pupil, nose, lips and other facial and body features.

Hi Asher,

Potentially, yes. However, the focused input signal will be blurred by the optical path (lens/aperture/OLPF/sensel aperture), symmetrically (with a good lens) in all directions. So it would require deconvolution sharpening to restore the original signal, which is then sampled slightly denser/sharper diagonally. Without deconvolution, or outside the DOF zone, it will just sample the blur more accurately.

Cheers,
Bart

 

[Doug Kerr]

The presentation above helps to show why the vision of a CFA array having an "effective pixel pitch" greater than the sensel pitch/delivered image pixel pitch is not fully useful.

Consider the following:

• Human vision is substantially more "sensitive" to the resolution of luminance than the resolution of chromaticity. (This is why chrominance subsampling in JPEG is effective, or the I-Q bandwidth conservation scheme in the NTSC video encoding scheme.)

• The G component of the sensor output is predominant in determining luminance.

• The G layer of a Bayer CFA array has the same sampling pitch (for detail along the horizontal or vertical axis) as the sensel array overall (even though the G array is "sparse").

This is why we are able, in visual resolution tests (by trumpet pattern, Siemens star, etc.) to note resolutions that are on the order of 75-85% of the (sensel-pitch-basis) Nyquist frequency.

In fact, in Bayer's original patent description, there were "luminance" photodetectors and two kinds of "chrominance" photodetectors - Y,Y,C1,C2). In later versions, with the R,G,G,B repertoire, the "G" photodetectors are still described in some places as "luminance" photodetectors.

Best regards,

Doug