Comment: Why the new "Dual Pixel AF" will transform the photo industry

Image ©Canon Europe. The Canon Dual Pixel CMOS AF sensor structure.

I do not normally comment on developments I have no inside information about. It is just adding to the noise in the internet. Here, I decided to make an exception :)

So, Canon launched the Canon 70D digital SLR camera 2013, July 2, two days ago. It includes an autofocus technology named "Dual Pixel cmos AF" which promises to introduce high-speed autofocus performance to live view shooting and movie recording which previously would have required a phase detect AF module, at least in lower light situations.

I believe that this innovation looks much more innocent than it actually is. Here is why.





A new kind of AF?


In the following, I will refer to this type of AF as "Multi Pixel AF" or short MPAF. The traditional two types are contrast detect AF (CDAF) and phase detect AF (PDAF). I will assume the reader to be familiar with how CDAF and PDAF works.

On July 2010, 21, Fujifilm introduced the F300EXR with a then new kind of AF, called "Hybrid AF" by Fuji. It makes use of the fact that the microlens which modern cameras have in front of every pixel on the sensor, creates a directional dependency of brightness distribution on the light sensitive pixel surface.

So, Fuji masked one half in some, the other half in some other of the pixels (but modified only a few pixels in total). It could therefore create two signals (low resolution images) from the modified pixels which are used to determine a phase shift between the two. The modified pixels aren't used to take the image itself.

Image ©Fujifilm.

Canon took this idea, sat down for a few hours and thought: "well, what if we didn't mask one half of a pixel but split it into two and only reading one at a time?" Here is what it looks like:

Image ©Canon.de. Canon replaced the mask by a second pixel which they call photo diode. When not doing the AF measurement, both pixels are combined to take the ordinary image. This is why every pixel on the sensor can become AF pixel, not just a few.

Obviously, Canon pays better graphics designers than Fuji. Other than that, the two illustrations look pretty similiar. So maybe, it is no surprise that it took them exactly 12 working days to write this idea down and file another patent on 2010, August 9, a Monday ... 2 busy weeks. The idea is obvious. But only Canon reacted quick enough to make it a patent. And only Canon was brave enough to actually double the pixel count in a real product. It took them 3 years to do. I applaud Canon for what I think is a business master piece.
(Disclaimer: the above sequence of events is deduced by me and assumed likely, it is not a fact!)

I found the relevant information in the Japanese internet:

• Canon patent "Image plane phase difference AF that does not degrade the image quality"
  Patent Publication No. JP 2012-37777 (-> PDF (jap.))
  Publication date 23 feb 2012
  Filing date 9 aug 2010
• Source: http://egami.blog.so-net.ne.jp/2012-03-21
• Discussion: http://digicame-info.com/2013/06/eos-70ddual-pixcel-cmos.html
  Interestingly, the discussion was 4 days before the 70D launch but nobody outside Japan seemed to have taken notice of ;)
• Later patent (algorithm) originally filed 13 dec 2011 (-> PDF (engl.))

Let me add a few last technical remarks. The dual pixel AF is reported by an insider to split the pixel into top and bottom half, yet all Canon marketing material shows it as left and right. Assuming top/bottom is correct, we end up with two different images from either half if out of focus. Both halves are blurred and shifted against each other by a few pixels. Both the blur (to be minimized, read contrast to be maximized) and the shift (which is the AF phase shift) can be used to compute a better if not optimum focus position. The following image illustrates it:

Image ©Canon.com. The middle shows the dual out-of-focus images (look at the curves below and their shape).
The right shows the in-focus images (they coincide and the curves are both sharper and unshifted).
For perfect focus, both, contrast maximization and shift minimization, can be combined.

I call it MPAF (multi pixel AF) because an improved version would use more than two pixels (think four or Quad pixel AF) or varying orientation to get rid of the orientation preference, not just top/bottom. It is like moving from line type to cross type AF sensors in classical PDAF.


Yes!

Yes, it really is a new type of AF. The idea is quite simple and is the same as with the Hybrid AF.
But executed with brute force.
And what you get in return, wow!

Unlike PDAF and Hybrid AF, CDAF can make use of all available light falling into the lens. This is a massive advantage. Sadly, it is offset by a large number of measurements it must take in order to maximize contrast. Especially as many vendors don't apply sophisticated algorithms to minimize the number of measurement steps. On the other hand, PDAF and even more Hybrid AF, only use a fraction of available light for their measurement.

The new MPAF is the first kind of AF which does both: use all light and minimize the number of measurements. Canon's first installation with its focus on continuous movie focusing may not tell the full story yet, but I am convinced that eventually, MPAF will blow all other kinds of AF out of the water. In terms of speed, tracking ability, accuracy, and low light capabilities.


And why it will transform the industry

What always was the advantage of mirror-based cameras (its better auto focus abilities) all of a sudden becomes obsolete.

Up to now, the mirrorless segment played entry level prosumer market. Because of a lack of competitive high end performance, there was no other choice. As a consequence, the choice of lenses and their image quality for mirrorless cameras left much to be desired. And because of the low price point, there was a natural limit of what electronic view finders (EVFs) could offer too (the other reason to keep the mirror). So, mirrorless vendors place a lot of emphasis on the form factor. Except that light sensitivity needs a certain lens diameter, whatever be the bodies' or sensor's form factor.

And so in 2012, the compact camera market collapsed into half, the mirrorless market shrunk considerably, even SLR sales was less and only the full frame camera market saw a solid growth. In a nutshell, the camera market sees a division into two ends: the commodity end driven by better and better smartphone cameras (e.g., iPhone 5 or Pureview) and the high end. The middle segment erodes and this started to hit the mirrorless idea. Which is a shame.

The MPAF is a breakthrough here: it eventually allows the high end cameras to drive innovation rather than play the conservative watcher. This redefines the rules and will transform the industry.

Only now it is a credible scenario that a Nikon D4 or Canon-1D X class camera becomes a mirrorless full frame with high performance MPAF, retina resolution 12bit lag-free EVF (a $1000 unit by itself), 24fps burst for stills and 4k and slow-motion video modes. With high-resolution $3000 wide angle lens options to rival medium format etc. Where new technologies can tickle down again.

Hope you enjoyed the read,
Falk


Update 2013, August 2

A new video emerged showcasing in a real life situation what the Dual pixel AF can do (and cannot do):

• Video 1 , Video 2

And Canon published an interview with the developers:

• Canon Whitepaper (PDF)