LumoLabs: Busting the alignment myth

This will be a special feature for hard core falconeye followers ;)

It all started with the exciting new feature of the Pentax K-7 camera which can produce true HDR images in-camera (it takes 3 shots, 3 exposure values apart from each other, and creates a single, tone-mapped or blended image from the sequence). It does it so well, esp. in the medium setting (as opposed to strong) that a limitation of the feature does actually hurt: you require a tripod to use the function.

The general opinion why Pentax left out the alignment step from their HDR function is that it is a processing power-hungry computation not easily done in-camera.

Not my opinion, though. I felt that Pentax has left out 10% of the implementation which reduces 90% of the functionality. Shouldn't it be the other way round?

Now, opinion isn't proof.

So, I implemented my own alignment operator, benchmarked it and compared its quality against the state-of-the-art in HDR.


The algorithm

The implementation is based on an algorithm whose computing time increases linearly with image size and is constant with amount of distortion. Note that I did not attempt to cope with the alignment problem known in panorama stitching, i.e., I don't do SIFT key extraction, I don't correct for lens distortion, I don't project onto a sphere before alignment, etc. pp. However, what I do is correct for shift and rotations which is a mathematical first order approximation to the latter and rather exact for small shifts and rotations.

The implementation uses a monochrome quad tree and high bit depth. The coding is in Java. It isn't based on anybody else's intellectual property.


The benchmark

I benchmarked the algorithm with a 5 image HDR from Munich Siegestor. It had distortions of up to 20 pixels and needed rotation as well. The source images haven't been ideally sharp (e.g., 1/25s and wide open with 15mm). And no program created a perfect alignment. So, I considered this example be adequate. The images were 14.6 MPixel JPG images.

The processing times on a Mac Mini (Early 2009), one processor used, are as follows:

• Single alignment: 175 ms
• Alignment of all HDR images: 700 ms
• Reading of all images from disk into Java heap: 16 s
• Shift-turn and write images to disk: 34 s
• Total, disk to disk (70 MB transferred), 5 images: 50 s

The K-7 has a dedicated image DSP and can certainly rival this. E.g., it can JPG-compress a 14.6 MPixel image in about 100 ms only!

Because the HDR function within K-7 already has created the required image structures in its 2 GB memory, it would be conclusive to assume that addition of an auto-alignment feature to the HDR function would add less than an extra 1/2 second to overall operation time.

If processing speed isn't the reason, so maybe it is quality? This must be looked at. In particular as some of the alignment operators I compared with took several to nearly 10 minutes to complete.

The quality

I've run several alignment operators and come up with the following ranking:

1. Photoshop CS3; PhotoAcute ("normal alignment")
2. Falk Lumo Operator (as of this blog article); PhotoMatix 3 ("by matching features")
3. PhotoMatix 3 ("by matching features"; "reduce ghosting")
4. PhotoMatix 3 ("by correcting horizontal and vertical shifts")
5. No alignment

All pre-aligned images have been read into Adobe Photoshop CS3, either by HDR merge (without further alignment) or by opening the 32 Bit .hdr-file written by PhotoMatix. Tonemapping was done in Photoshop CS3, using the "Equalize Histogram" method. The latter isn't best but as it has no parameters, I thought it would make for a good option in a comparative study. Finally, the result was studied to derive the above ranking. Which is entirely subjective!

Below are the two samples from category #2. Please, visit the gallery to view the other samples, too. Note that all sample images have been resized to 6 MPixel.

Fig.1 No alignment


Fig.2 PhotoMatix


Fig.3 Falk Lumo Alignment Operator (as decribed here)

Take me to the gallery

Conclusion

The myth is busted. There is no reason to not include auto-alignment into the next firmware release of the K-7 HDR feature. In less than an extra second of processing time, about PhotoMatix quality can be provided. Make it so!