July 28, 2009

LumoLabs: Busting the alignment myth

This will be a special feature for hard core falconeye followers ;)

It all started with the exciting new feature of the Pentax K-7 camera which can produce true HDR images in-camera (it takes 3 shots, 3 exposure values apart from each other, and creates a single, tone-mapped or blended image from the sequence). It does it so well, esp. in the medium setting (as opposed to strong) that a limitation of the feature does actually hurt: you require a tripod to use the function.

The general opinion why Pentax left out the alignment step from their HDR function is that it is a processing power-hungry computation not easily done in-camera.

Not my opinion, though. I felt that Pentax has left out 10% of the implementation which reduces 90% of the functionality. Shouldn't it be the other way round?

Now, opinion isn't proof.

So, I implemented my own alignment operator, benchmarked it and compared its quality against the state-of-the-art in HDR.

The algorithm

The implementation is based on an algorithm whose computing time increases linearly with image size and is constant with amount of distortion. Note that I did not attempt to cope with the alignment problem known in panorama stitching, i.e., I don't do SIFT key extraction, I don't correct for lens distortion, I don't project onto a sphere before alignment, etc. pp. However, what I do is correct for shift and rotations which is a mathematical first order approximation to the latter and rather exact for small shifts and rotations.

The implementation uses a monochrome quad tree and high bit depth. The coding is in Java. It isn't based on anybody else's intellectual property.

The benchmark

I benchmarked the algorithm with a 5 image HDR from Munich Siegestor. It had distortions of up to 20 pixels and needed rotation as well. The source images haven't been ideally sharp (e.g., 1/25s and wide open with 15mm). And no program created a perfect alignment. So, I considered this example be adequate. The images were 14.6 MPixel JPG images.

The processing times on a Mac Mini (Early 2009), one processor used, are as follows:

  • Single alignment: 175 ms
  • Alignment of all HDR images: 700 ms
  • Reading of all images from disk into Java heap: 16 s
  • Shift-turn and write images to disk: 34 s
  • Total, disk to disk (70 MB transferred), 5 images: 50 s

The K-7 has a dedicated image DSP and can certainly rival this. E.g., it can JPG-compress a 14.6 MPixel image in about 100 ms only!

Because the HDR function within K-7 already has created the required image structures in its 2 GB memory, it would be conclusive to assume that addition of an auto-alignment feature to the HDR function would add less than an extra 1/2 second to overall operation time.

If processing speed isn't the reason, so maybe it is quality? This must be looked at. In particular as some of the alignment operators I compared with took several to nearly 10 minutes to complete.

The quality

I've run several alignment operators and come up with the following ranking:

  1. Photoshop CS3; PhotoAcute ("normal alignment")
  2. Falk Lumo Operator (as of this blog article); PhotoMatix 3 ("by matching features")
  3. PhotoMatix 3 ("by matching features"; "reduce ghosting")
  4. PhotoMatix 3 ("by correcting horizontal and vertical shifts")
  5. No alignment

All pre-aligned images have been read into Adobe Photoshop CS3, either by HDR merge (without further alignment) or by opening the 32 Bit .hdr-file written by PhotoMatix. Tonemapping was done in Photoshop CS3, using the "Equalize Histogram" method. The latter isn't best but as it has no parameters, I thought it would make for a good option in a comparative study. Finally, the result was studied to derive the above ranking. Which is entirely subjective!

Below are the two samples from category #2. Please, visit the gallery to view the other samples, too. Note that all sample images have been resized to 6 MPixel.

Fig.1 No alignment

Fig.2 PhotoMatix

Fig.3 Falk Lumo Alignment Operator (as decribed here)

Take me to the gallery


The myth is busted. There is no reason to not include auto-alignment into the next firmware release of the K-7 HDR feature. In less than an extra second of processing time, about PhotoMatix quality can be provided. Make it so!


  1. Great job! Why don't you offer your algorithm to Pentax? Maybe in exchange for a camera or two...

  2. Wow, I'm impressed. I think Pentax engineers got blush after reading this...


Please if posting anonymously, choose a nickname for your post. Thanks.