• Please use real names.

    Greetings to all who have registered to OPF and those guests taking a look around. Please use real names. Registrations with fictitious names will not be processed. REAL NAMES ONLY will be processed

    Firstname Lastname

    Register

    We are a courteous and supportive community. No need to hide behind an alia. If you have a genuine need for privacy/secrecy then let me know!
  • Welcome to the new site. Here's a thread about the update where you can post your feedback, ask questions or spot those nasty bugs!

The "dual pixel raw" output of the Canon EOS 5D Mark IV

Doug Kerr

Well-known member
Introduction

The Canon EOS 5D Mark IV, like several Canon DSLR cameras before it, utilizes "dual pixel" technology. That means that each sensel (these occupy the positions that will correspond to image pixels at the camera's highest resolution setting) in fact comprises two "subsensels" (side by side). The working of the microlens structure in front of the sensels results in one one of the two subsensels receiving light from approximately the left half of the lens' exit pupil and the other of the two subsensels receiving light from approximately the right half of the lens' exit pupil.

We see that this is very much the same, for any given sensel, as the arrangement of the "phase-comparison" autofocus detector. There, each AF detector has two little sensors, one of which receives an image formed through a small sub-aperture located at one edge of the lens' exit pupil and the other of which receives an image formed through a small sub-aperture located at the opposite edge of the lens' exit pupil.

The overall principle is that of a rangefinder (no, I don't mean a "rangefinder camera", although such has a rangefinder in it). The two overlapping images of the scene we see trough a rangefinder (generated through two lenses separated by some distance) are displaced from each other by an amount corresponding to the difference in the distance to that part of the scene from the current distance setting of the rangefinder. When the distance setting is in fact the distance to that part of the scene, the two overlapping images are exactly coincident.

Similarly, in the phase comparison AF detector, the images of some scene element on the two little sensors for a given AF detector are displaced from each other by an amount that is related to the current error in the focusing of the camera. That displacement is determined by image correlation technique and used to guide the "refocusing" of the camera.

We can see that in a "dual pixel" camera, the entire sensor (at least the amount that is fitted with "split sensels") comprises two sensors of a phase-comparison autofocus detector. Thus we can have continuous focus error detection (with respect to as many "AF points" in the scene as we care to provide implementation for) during "live view" or "video" shooting modes.

But with regard to the actual capture of the image, the digitized outputs of the two subsensels at each sensel position are added together to get what is considered, by further imaging processing, the "output" of that sensel.That is, the "whole sensels" are used for actual image generation. At least in Canon dual-pixel cameras prior to the ESO 5D Mark IV.

A new wrinkle in the Canon EOS 5D Mark IV ("5D4") is an optional "dual pixel raw" output.

Before we proceed, we note that the two subsensels at each sensel position are spoken of as the "A" and "B" subsensels. Their analog outputs, by the way, are separately digitized.​

When we enable the "dual pixel raw" feature, the raw output file includes two complete arrays of sensel raw output values. One, spoken of as the "main subframe", carries for each sensel location the "consolidated" digital sensel output, derived as the sum of the two digitized subsensel outputs (spoken of as the "A+B" value). The second array, spoken of as the "auxiliary subframe", carries only the digitized output values of the collection of "A" subsensels.

And we of course recognize that software working with this raw data can readily determine, for each or every sensel, the output value of the "B" subsensel (by subtracting the "A" value from the "A+B" value).​

Making use of this "dual pixel" raw information

Canon tells us that in the latest version of their Digital Photo Professional (DPP) image processing software, this enlarged collection of data in a "dual pixel raw" file can be used to achieve the following (and I will quote directly from the Canon advertisement:

• When an image isn’t quite sharp enough in a specific area, such as on a subject’s eyes for example, it’s possible to adjust the image and bring sharpness back to a desired area.

• With Dual Pixel RAW it is possible to adjust the position of out of focus areas, or ‘bokeh’ to enhance composition, for example moving an out of focus object that is obscuring the subject.

• Unlike lens flare, which can sometimes be used creatively in making an image appear low in contrast and atmospherically ‘hazy’, ghosting is not desirable and shows internal lens reflections. This can be controlled with Dual Pixel RAW and minimized with careful adjustment.

I have not yet given any thought to just how any of these work technically. (I hope to look into that as time here allows).

The Canon advertisement to which I refer is available here:

http://www.canon.co.uk/cameras/eos-5d-mark-iv/dual-pixel-raw/


Another implication of the "dual-pixel raw" output

Will Thompson was kind enough to refer me to an interesting article on the site "RawDigger", whose masthead motto is "Everything You Always Wanted to Know About Raw
(But Were Afraid to Ask)". The article is here:

http://www.rawdigger.com/howtouse/canon-dual-pixel-mode-highlights-are-there

The issue at hand is the matter of the "clipping" of "overexposed highlights". We will assume that we are not operating at the sensor's "native" ISO sensitivity but rather at some higher ISO sensitivity. Thus, such "clipping" occurs not when the photometric exposure on one or more sensel photodetectors would "saturate" the the photodetector itself, but rather when that photometric exposure would deservedly produce a digital number (DN) value greater than the maximum limit of the format used to record the DN.

In the EOS 5D Mark IV, that number is recorded in 14-bit form. The highest possible 14-bit number is 16383. and evidently, from the article, values up to that number are allowed.
This is not so in all Canon cameras; sometimes the highest "used" DN value is made less than the mathematical upper limit of the number format.​

So, the digitized output of either an "A" or a "B" subsensel could be (we will assume) as high as 16383. But in the "normal" processing of the sensor data, and in the "main subframe" delivery of the raw data, where we work, for each sensel, with A+B, the highest allowable value of A+B is also 16383.

Now, the working of the automatic exposure system obviously must try to keep the photometric exposure on the sensor over the whole image so as to minimize the prospect that A+B would be over 16383. Since, in general, the outputs of the two subsensels of a sensel would be very nearly equal, that means that the exposure control system would aspire to have very few if any subsensels whose digital output ("A" or "B") would be over 8191.

But of course that doesn't always work. Suppose that in some highlight region for a certain sensel the digitized outputs of the two subsensels are both 8300. Then A+B would be 16600. That is of course not a permissible value for the consolidated output, for which the limit is 16383. Thus, the value of that sensel used in development of the image would be 16383, not representing the true luminance of the highlight - the highlight has been "clipped".

But, the author points out, if we choose to have an image developed from the "auxiliary subframe" (which carries the "A" subsensel values only), in that highlight the encoded value (DN) of our example "overexposed" subsensel is 8300, a perfectly valid 14-bit number. Thus we can develop an image with the highlight region not clipped.

Now obviously some attention must be paid to the fact that all the DNs in this suite of raw value are working on a scale half the size of the normal scale, so some attention needs to be given to attaining the right overall tonal scale. Whether the DPP raw development software will somehow do that automatically, or whether we have to do it with the "exposure" slider, I don't know.

So what's going on here? Well, simplistically, we can describe this way: If after a shot, we realize that (for normal processing) we have highlights blown out, we can in effect get a "redo" corresponding to using 1/2 the original ISO sensitivity, and then develop the image from that set of raw data. The increased "margin" agist clipping is on the order of one stop.

But, oh, by the way: This "redo" is done with sensel photodetectors having half the area we would normally use, so we can expect a deterioration in noise performance, probably manifested in visible noise in the darker areas.

So there is no free lunch. But there is a "second chance".

About "dynamic range"?

Some readers of the article may think, "Can we use a similar ploy to get increased dynamic range in a 5D4?" The thinking might go like this: "Well, after all, by using this ploy we get about one stop of greater highlight "headroom". Isn't that the same as an increase in dynamic range?"

Keep in mind that by using the "ploy" we get an additional stop of highlight accommodation for a given aperture, shutter speed, and ISO sensitivity. We could, when using the traditional form of raw data, have gotten the same increase in headroom by using a smaller aperture, a shorter exposure time, or a setting of the ISO sensitivity of one half the original value. (It is in fact a reduction in the ISO sensitivity value that best corresponds to what we do, in the "redo", using the auxiliary subframe.

And of course none of those would constitute attainment of greater dynamic range.

We need to be mindful of how "dynamic range" is defined - and of course many definitions are used. The one most widely accepted today is that the dynamic range of a camera is the ratio of (a) the greatest phtometric exposure for which a distinctive digital output is delivered to (b) the least photometric exposure for which the signal-to-noise raio (SNR) of the delivered "image" is at least 1:1.

We recognize that, for any given photodetector design, for a given photoelectric exposure, the SNR varies with photodetector area. Thus, if we were to cut the size of the photodetectors in a camera in half, keeping everything else the same, the dynamic range would perforce decrease.

Now, as we consider moving from the "normal" use of raw data to the "auxiliary subframe" mode, we have in effect reduced the actual ISO sensitivity of the sensor by a factor of 2, and have reduced the area of the sensel photodetectors by a factor of 2. Thus we could expect a reduction, not an increase, in dynamic range from using the "auxilary subframe" to produce our image.

I think.

Best regards,

Doug
 

Doug Kerr

Well-known member
It is interesting to compare the two images developed from the same raw file presented in the RawDigger article.

First I will show the two images, first the developed from the main subframe ("A+B") and then the one developed from the "auxiliary subframe ("A").

RawDigger_91A0045-20160901-123750-RawDigger-ScreenShot_frame1.jpeg


from RawDigger​

RawDigger_91A0045-20160901-123708-RawDigger-ScreenShot_frame2.jpeg

The sky is the primary area where we have clipped highlights in the first image.

You can probably recognize visually that the second image is "darker", the result of each sensel output being half what it would be in the first image.

We can confirm this by looking at the two histograms kindly provided by RawDigger:

RawDigger_91A0045-Full-6744x4502_frame1-Histogram.jpeg


RawDigger_91A0045-Full-6744x4502_frame2-Histogram.jpeg

We can easily see that the second image is about "one stop darker" than the first, just as we would expect.

Now, can we "boost" its brightness? Well, if done in a simplistic way, not without again "blowing out" the highlights in the sky. (Of course, there would be many techniques for increasing the overall visual lightness of the image while still retaining detail in the sky.)

Time for breakfast.

Best regards,

Doug
 
Top