(Photo ©2009 Karsten Pawlik.)
After the interesting findings about the relatively good dynamic range properties of the Pentax K-x (cf. Lumolabs: Pentax K-x sensor quality), many have requested to publish a comparison with some other well known camera models.
Well, I am happy to announce that a great photographer and dear friend of mine, Karsten Pawlik (kafenio.org) shares my passion and we could spend some time in studio together in order to compile this comparison.
Most of the photos (e.g., all Nikon photos) presented in this article are © Karsten Pawlik. The opening photo shows two Pentax K-x with the gorgeous Pentax FA 31/1.8 Limited lens, in both black and silver color.
It is interesting to compare the K-x with these two cameras from Nikon for the following reasons:
- Nikon D5000 and Pentax K-x may share the same Sony Exmor sensor which is supposed to be found in the Sony A500 too. (well, maybe not: DxO measures that the A500 has a different sensor)
- Nikon D700 is a full frame camera and captures 1.2 stops more light than both the Nikon D5000 and Pentax K-x which are both APS-C only. It is interesting to see if this advantage shows in images.
1. ISO Noise
(please click on the photo above (and all others) for better readable versions)
The above chart summarizes the noise levels for 18% gray patches (2800 °K tungsten) at various values for ISO: 100, 1600 and 12800, measured as a signal to noise ratio (SNR [dB]). Please read Lumolabs: Welcome and testing methodology for details about how we measure these values. Towards the end of this article, you'll find links to all original test shots suitable for further examination.
Within this article, I will ignore the fact that the D5000 delivered the best SNR at ISO 100. AT ISO1600, we see the expected result: About 3dB or 1 f-stop (1EV) advantage for the full frame D700 with respect to both D5000 and K-x.
At ISO 12800, the K-x manages to match the result of the D700 while the D5000 behaves as expected. However, the SNR should decrease by at least 9dB when going from ISO 1600 to 12800 while it only decreases by only about 6dB for the K-x. This is a sign for raw file noise reduction at work for the Pentax and I believe that all SNR values for Pentax K-x should be reduced by 3dB for ISO values of 3200 or higher.
(Note: The D5000 has no native setting of ISO 12800. It was emulated using ISO 6400 and exposing like for ISO 12800, pushing 1 stop in postprocessing.)
In summary, we confirm the 1 stop advantage for the full frame sensor as far as high iso gray level noise is concerned.
2. Dynamic range
The situation turns out to be more complicated when we study the respective dynamic ranges. Let's start with the full SNR curves for all cameras.
Look first at the three red (D700 full frame), blue (D5000) and green (K-x) curves in the middle of the graph: These are the ISO 1600 curves for the three camera models and the red line lying above the others shows the full frame advantage we have been talking about above. The curves show how the signal-to-noise ratio decreases (relative noise increases) towards the shadows (the lower luminosities towards the left). At ISO 1600, the full frame advantage remains significant throughout all luminosities spanning 7 stops (1%). Note that the curves at ISO12800 may be "polluted" by noise reduction tricks for the APS-C sensors.
Most important for the determination of dynamic range are the SNR curves at lowest ISO (i.e., ISO 100). In order to measure deepest shadow noise, we made three shots at ISO100: normal, underexposed by -5EV and underexposed by -10EV! Patches from all three shots were analyzed to compile an SNR curve spanning 13 stops (down to 0.02% luminosity (which is RGB 0.05/255 and sRGB 0.6/255 aka perfect black)).
The surprising result is that at deepest shadows, the D5000 outperform the D700 by 2 dB and the K-x by even 5 dB. This should translate to about 0.5 EV more dynamic range for the D5000, and about 1.5 EV more dynamic range for the K-x, when compared to the D700. DxO does indeed confirm more dynamic range for the D5000 (12.5EV) compared to D700 (12.2EV). In combination, this is a confirmation for the "13EV or better" dynamic range claim I made in my earlier blog article.
3. Dynamic range (revisited)
I decided that this isn't the full story. Numeric results of measurements of noise are one thing. Plausibility checks are another.
(you must click the image and then select "O" (original size) to use this chart!)
The above chart contains various (small) noise patches from all cameras and ISO values. I even added some Pentax K-7 patches for comparison (Note: the K-7 has more pixels). Towards the end of this article, you'll find links to all original test shots suitable for further examination.
A quick examination of the patches at ISO 100 and -10EV shows that the D700 wins hands-down: better color accuracy, better readability of letters and better contrast. It sure looks slightly more busy (more noise) but the noise looks more fine grain too. More fine grain? Yes, the noise from the D5000 and K-x look coarser than the noise from the D700 or K-7. But noise should be statistically uncorrelated and look always fine grain (except for pattern noise).
Good detail note: all three cameras have no visible pattern noise.
Obviously, the sensor of the Nikon D5000 and Pentax K-x (and I guess it is the sensor rather than the firmware) plays some tricks with the signal in deepest shadows: there seems to be some sort of deep shadow noise reduction at work (like binning dark pixels) which leads to softer detail and a lower noise measurement.
Therefore, I conclude that the excellent dynamic range measurements of the D5000 and K-x are partly due to sophisticated signal processing, possibly on the sensor itself.
However, even taking this disclaimer into account, we can say that the deepest shadow reproduction of all three cameras play in one league: e.g., look at the 153 letters in the -10EV samples and compare to the K-7 (-9EV only) where the letters aren't legible anymore.
I conclude that by dynamic range, taking all information into account, the cameras rank as follows:
- Nikon D700 (#3 by lab test)
- Pentax K-x (#1 by lab test) and
Nikon D5000 (#2 by lab test)
and are all very close.
The ISO12800 patches roughly confirm the numeric lab results. A K-x at ISO12800 and a K-7 at ISO6400 do roughly have the same noise while the K-7 keeps a slight resolution advantage even at such high ISO values.
4. Conclusion
Traditionaly, there is a one stop difference between full frame and APS-C. Temporarily, this sensor (Nikon D5000, Pentax K-x, supposed to be a Sony Exmor sensor) fills the gap (towards a Nikon D700) with respect to dynamic range while the difference remains with respect to high ISO noise.
This means that the D5000/K-x sensor has the same sensitivity to light as usual (as the D700) but has significantly reduced read-out noise.
The Pentax K-x may have a slight advantage over the Nikon D5000, esp. at very high ISO values. At the same time, this sensor (or firmware library) seems to include raw data alteration which makes the numerical analysis even more difficult, even when done in the DxO way. The time for true signal measurements has come, defining random noise as the sample variations of the signal and fixed pattern noise as the difference of the mean value with the ideal signal.
Lab tests (our own and the ones published by DxO) make the D5000/K-x have better dynamic range than the D700. But this is due to the limited testing methodology. D700 still has the better dynamic range, but by a small margin only.
5. Further reading
Access to full sample shot material:
- Pentax K-x test shots (full size)
- Nikon D700 test shots (full size)
- Nikon D5000 test shots (full size)
- Lumolabs: Welcome and testing methodology
Enjoy! :)
sorry, but your findings are wrong ! For example: Your diagram shows for the D5000 and 1% gray a SNR value about 32dB, but DxoMark shows a value of about 24dB. Maybe you are not able to read diagrams correctly. Please learn this now and please correct your nebulous findings !
ReplyDelete@Anonymous, why do you think that DxOLabs results have to be the same?
ReplyDeleteI reproduce part of their testing methodology for the SNR curve but use different color temperature and different data than them. E.g., I use demosaiced data from a process which I use as my standard (follow the forth link in my article's appendix for a description of my testing methodology). DxOLabs doesn't publish how they derive their input data but most probably they use data prior to demosaicing.
I am able to read diagrams correctly and more.
My findings are less nebulous than DxOLabs'. Feel free to run your own analysis software across sample charts as they are all online (unlike DxOLabs results which cannot be reproduced by independend parties -- on purpose, I guess, because they sell their analysis software).
My process and publication method follows the principles of scientific routine meant to be verified or falsified by other serious workers.
It is all online. Do your scientific homework and reply with a more qualified comment if your own findings are worth it. Thank YOu.
Does DxO publishs full SNR data, btw? (They show only 18% SNR?)
ReplyDeleteAnd yes, different tests do not have to produce the same test result, as they are *different* tests! :-)
Thank you for sharing your findings with the community! I am currently searching for a replacement for my trusty k100d super and had my eyes set on the k7. Now I'm wondering whether a k-x might do, given the excellent results.
ReplyDelete@RiceHigh, yes, the Full DxO curves are selectable for display when looking at a SINGLE camera's results. This option is absent in compare mode, though. Scientific tests have to be reproducable. This is the heart of science. But DxO doesn't publish enough information to be able to reproduce their data input into statistical analysis. Which is why results are expected to differ. My test results can be reproduced by any interested party. So, if I made errors, they can be found by others.
ReplyDelete@Anonymous, the K-7 is the better camera. The K-x has about 1 stop less read-out noise. You must decide how much of your photography depends on read-out noise. Moreover, the K-7 has less noise in video (using more pixels). I own both cameras and most of the time, grab the K-7 :)
@Falk Lumo: Thanks again for the tip (and your extensive k7 beta test). Sometimes, it's hard to tell from measurements how big the difference will be in everyday photography. I plan to have the camera with me as often as possible. That's why I stayed with Pentax from MX days of yore: They do produce compact gear - and the best camera won't be any good when it's left at home, because it's too bulky. The k7 combined with the DA 40 will be a nice and light package, just like the k100d super.
ReplyDeleteWow, good stuff. I was contemplating buying the D5000 but now the L-x is a definite possibility.
ReplyDeleteon a technical ground: you say "the SNR should decrease by at least 9dB when going from ISO 1600 to 12800". Why is that? I understand that 9dB is 3 stops, or eight times of a difference. And with 8 times higher sensitivity, you decrease the shutter time by a factor of 8, so the input signal for the pixel (defined as a number of photons) is obviously 8 times lower. But why do you assume that the input-referred noise is the same or higher for higher sensitivity? In fact, it this was true, than increasing the sensor sensitivity when taking photo would not make any sense - just underexposing would be a better option.
ReplyDeleteI would rather think that when going from ISO 1600 to 12800, the SNR should increase by less than 9 dB (with no noise reduction applied). If you compare results for ISO 100 and 12800 (21dB increase), the D700 and K-x show SNR decrease of say 18 and 17dB, respectively. But the d5000 shows 23dB of SNR decrease. Does it mean that for the d5000, photo taken with ISO100 and underexposed by 7 stops (and then corrected for exposure in RAW developer) would have lower noise than photo taken with sensor sensitivity set to ISO12800, with the same aperture and shutter time combination??
@Pawel, good observation and therefore, let me explain: the 3dB/stop rule holds true for pure photon shot noise. For higher sensitivities, the read-out noise means that the decrease is faster (steeper curve). For lower sensitivities, limited well capacities can mean that the decrease is slowlier (flatter curve). But for a sensor with base ISO of 100 or 200 and between ISO 1600 and 12800, it nowhere should be flatter than 3dB/stop. If it is then noise reduction must be at work.
ReplyDeleteAddendum to my answer to Pawel: I don't assume constant noise. Assuming constant noise would lead to 6dB/stop lower SNR. Pure photon shot noise leads to 3dB/stop, a sum of photon shot noise and read-out noise leads to anything between 3db/stop and 6dB/stop.
ReplyDeleteAnother addendum because not all people understand this: Read-out noise affects both dark shadows and high sensitivities. Read-out noise has two terms, one independent of ISO gain and one dependent on ISO gain. Switching the camera to a higher ISO setting helps reducing the read-out noise between the gain amplifier and the A/D converter.
ReplyDeleteFalk Lumo,
ReplyDeleteThanks for the answer. Now some things are more clear - first, why do you expect 3dB SNR decrease for one stop higher sensitivity, and not 6dB :) For shut noise it is obvious. I was a bit surprised that your SNR analysis is mostly based on the shut noise, since this noise has nothing to do with sensor design. But I see from your plots that this in fact is the most important source of noise - please correct me if I am wrong, but if not, then the sensor technology is really close to fundamental limits! - so this methodology makes more sense to me now.
By the way, it should be possible to find theoretical level of shut noise for given combination of shutter time, aperture and pixel area, right? Could be fun to compare the real noise level curve (as you show in section "ISO noise") with such theoretical limit curve. Not sure, however, how to take into account the effect of the on-sensor color filter.
Anyhow, this is a very nice work that you are doing here.
@Pawel, from a precise measurement of the SNR curve, you can determine both read-out noise (can become zero) and photon shot noise. From the photon shot noise, you can compute the full well capacity and sensor cell efficiency. From sensor cell efficiency, fill factor and bayer filter spectral sensitivity, you can compute the quantum efficiency. Quantum efficiency can already be as high as 40% and is limited to 100% by the laws of nature. 100% quantum efficiency, no read-out noise and no Bayer filter describes the fundamental limit.
ReplyDelete@Falk Lumo:
ReplyDelete"the K-7 is the better camera. The K-x has about 1 stop less read-out noise. You must decide how much of your photography depends on read-out noise. Moreover, the K-7 has less noise in video (using more pixels). I own both cameras and most of the time, grab the K-7 "
Looking at the chart image you'd think the K-7 has the worst results... judging by the K-x results and thinking from a budget point of view would the K-x be a highly pleasing camera to buy still if I didn't want to spend too much??
@Anonymous, the K-x certainly IS a highly pleasing camera :) In this price range, the only other camera coming close is the Nikon D5000. But the K-x is smaller (even pocketable with the DA40 lens) and has in-body shake reduction.
ReplyDeleteBrilliant work thank you
ReplyDeleteThis clears a lot of questions that I have had
for some time.
One post here gives the K7 a thumbs up over the Kx
In the many photos I have viewed on line , I see far more noise with k7. yet photos I have seen on line the Kx have finer quality than many other cameras at a higher price point
your expert analyses of the K7 would be great,
it would clear up many questions a lot of Us have.
Not being able to see any Pentax gear leaves big doubt a lot of us have
Thank you for your excellent post
You'll find my comparison K-7 vs. K-x right here in my blog (back in October).
ReplyDeleteI am using my K-7 without any hesitation up to ISO 1600. It is true that the K-x produces similiar noise only at ISO 4800 and therefore, I am using my K-x without any hesitation up to ISO 4800 (K-7 ISO 3200 and K-x ISO 6400 are comparable again).
Between ISO 100 and ISO 1600 I prefer my K-7 by a large margin. It is a more professional tool, simply. And for somebody who used to shoot Kodachrome 25, ISO 1600 doesn't feel too bad ;)
This comment has been removed by a blog administrator.
ReplyDeleteReally like Nikon D700.
ReplyDeleteGreat post
Hey falk,
ReplyDeleteThanks for this test. Will you be a doing a similar test for k-5? Don't care that much for D500, but a comparison between k-5, k-x and D700 will be very welcome.
cheers,
Abhi
I wanted to thank you for this great read!! I definitely enjoying every little bit of it I have you bookmarked to check out new stuff you post.
ReplyDeleteDigital SLRs.
I've used the D5000, D5100, D700, and Coolpix A for a while, and one thing that I noticed was how I was consistently able to push the 200 ISO D700 RAW images more in LightRoom than ANY of the DX sensor RAW image files. Lot of other factors there such as processing, lens, and LR's algorithms perhaps, but what I'm able to do with the images seems consistent and even intuitive, and it's almost like the D700 RAW image file contains better color information in the shadows and highlights... which is not apparent when looking at just the RAW image files. Is this related to the sensor's Dynamic Range? Doesn't seem to correlate with DR lab test results, so I'm a little bit puzzled. One factor to point out is that the D700 usually has great glass, whereas the other cameras don't... but I'm not talking about sharpness here, perhaps the ability for the lens to transmit light and color.
ReplyDelete