• Global Sites

It's all about noise and how you deal with it.

THIS GUIDE IS FOR:

Technically proficient microscope camera users who want to understand what Huang, et al.1 did to achieve video-rate imaging speeds for super-resolution microsocpy.

THIS GUIDE OFFERS:

A discussion of the algorithms used by Huang, et al,1 that helps answer the question, "Do I need to use similar algorithms for my experiments?"

Learn more...

Learn more about the camera used by Huang, et al,1 the ORCA-Flash4.0 V2, read ORCA-Flash4.0 V2 product page.

Why did F. Huang, et al.,1 generate new software algorithms for implementing single molecule switching nanoscopy (SMSN) with an sCMOS camera?

The key is noise, and what noise sources are important.

First, the bottom line:
  • For CCD cameras, the only camera contribution to noise that is significant is from read noise—dark noise is negligible at typical exposure times for biological experiments.
  • For EM-CCD cameras running in EM mode, additional sources of camera noise become noticeable and need to be accounted for.
    • The EM register adds an extra noise factor—excess noise—that effectively reduces the quantum efficiency (QE) by 50% across all spectra
    • Fixed pattern noise and gain drift calibration also need to be considered in the total noise equation.
    • These noise sources are especially important for researchers using computational (statistical) imaging techniques, but researchers who’s goal is an accurate visual image also need to be aware that these sources of noise are present in their images.
  • For sCMOS cameras, we also need to include more than just read noise in our noise equation.
    • As with EM-CCD cameras, fixed pattern noise is non-negligible
    • Each pixel of the sCMOS camera has it’s own offset to account for dark noise—the offset for each pixel is measured and then factored into all future calculations.
    • sCMOS has more pixel-to-pixel variability—a consequence of the differences in sCMOS architecture from CCD architecture—which can be measured and factored into all future calculations.
    • Most researchers do not need to account for individual pixel offset or pixel-to-pixel variability. Pixel-by-pixel variability is only an issue for single molecule localization experiments, when high spatial accuracy and precision are required.
  • These additional sources of noise—for both EM-CCD cameras and sCMOS cameras—typically need to be accounted for only when working at the limits of the camera.
Second, the details

Total noise is typically calculated as:

total noise

where read noise and dark noise are contributed by the camera, and photon shot noise is a function of the statistical uncertainties in photon.

While the equation above is quite accurate for CCD cameras (i.e. ORCA-R2), which are quite “quantitative,” it does not accurately describe the main sources of noise in either EM-CCD or CMOS cameras. A far more comprehensive noise equation also includes fixed pattern noise and excess noise

total noise

When the equations start expanding to include all this detail it’s easy to lose sight of what they mean for your biology. Ultimately it comes down to making sure that the camera, image analysis, and biological question are all aligned so that your data is real and accurate. And that’s what F. Huang, et al.,1 have done here. Below we distill this tour de force paper and its implications for SMSN and other biological applications.

Where did all this noise suddenly come from?

Historically, SMSN experiments are performed using EM-CCD cameras, where, because of low dark current and relative read noise, the dominant source of noise is shot noise. But there are two often overlooked source of noise in these experiments—excess noise—which gives rise to the excess noise factor (F)2 and fixed pattern noise.

The impact of excess noise on localization precision was noted by both Raimund Ober’s group3 and Henrik Flyvbjerg’s team4 in 2010, and Pertsinidis, et al., carefully considered fixed pattern noise in their 2010 Nature article.5 Finally, a direct comparison between CMOS and EM-CCD for SMSN with excess and fixed pattern noise considered was presented by Zheng-li Huang in 2012.6 Ultimately each of these papers noted that considerations of the sensor, including pixel-to-pixel differences, enabled greater precision and accuracy in SMSN experiments.

Do we need to account for these noise sources when using CMOS technology?

With new technology comes new developments. Using sCMOS for SMSN offers much greater field of view and faster speeds and avoids the contribution of excess noise. However, with the different chip architecture of sCMOS technology, where each pixel has its own charge to voltage converter, all noise variations must be considered on an individual pixel-to-pixel basis. F. Huang, et al,1 take all these pixel-by-pixel differences, including gain (signal dependent and most notable at higher light levels), dark offset variation, and read noise (signal independent and most notable in low light conditions) into consideration and adapt their localization algorithms to account for it.

The sCMOS versus EM-CCD chip architecture difference highlights the strengths and weaknesses of these two unique technologies. Read noise is the collective term applied to noise introduced when photoelectrons are converted from a charge into a voltage by an amplifier, and then when that voltage is translated into to a digital number (or grey level) by an analog to digital converter.

EM-CCDs are essentially specialized CCDs. During read out, electron multiplication (EM) of the signal happens in an additional gain register that is on the chip but before readout. As the charge is serially transferred through this EM region, the signal is multiplied through a process called impact ionization. The charge is then readout as it would be for a CCD camera through a single amplifier.

Relative read noise in an EM-CCD is extraordinarily low (<1 e- rms), not because the read out amplifier is special, but because the signal is amplified prior to readout. In other words, EM-CCD read noise is reduced by a factor equal to the gain applied. In imaging, as in life, nothing is free. The low read noise is a wonderful feature of EM-CCD technology, but it comes at the cost of introducing a new noise source called excess noise, which depends upon the signal—it is proportional to the square root of the signal. Excess noise occurs in the EM register and is the noise associated with the multiplication process.

In contrast to EM-CCDs, sCMOS has no EM register, each pixel has its own charge to voltage converter, and each column (or half column) has its own amplifier and analog-to-digital converter. The former means that excess noise doesn’t contribute to the noise for sCMOS; the latter means that sCMOS readout can be faster than an EM-CCD because the pixel photoelectron-to-voltage conversion in each row happens all at the same time, in parallel for each column. The tradeoff is that now there is the potential for greater pixel-to-pixel variation in read noise. Our key spec for read noise in a sCMOS describes the rms value of the read noise distribution in a single frame…but there will be a distribution (see graph) and pixels that have a read noise different from the rms mean, in SMSN, can skew the localization of single molecules, and create artifacts in the image.

To account for these differences and achieve precisely localize molecules using an sCMOS camera, F. Huang, et al,1 had to modify algorithms to account for read noise, offset and conversion factor (gain) of each pixel. The first step of this process involved characterizing the camera at the pixel level. They conducted measurements with no incident photons (i.e., with the lens cap on) to understand camera performance in the absence of signal and with uniform light of varying intensity to characterize pixel response non-uniformity

They calculated the ADU offset for each pixel—the constant level of analog-to-digital units engineered into the readout process to prevent negative ADUs caused by read noise—and the pixel-to-pixel read noise. ADU offset is required to avoid erroneously interpreting the ADU offset as optical signal. The last part of this process involved using ADU offset and read noise variance to calculate pixel-to-pixel gain.

Calculating pixel-to-pixel gain then involves making measurements of the signal at many light levels, and then using the expected statistics (shot noise and read noise), along with the ADU to estimate the gain for each pixel. The best estimate of the gain is obtained by minimizing the difference between the measured gain and the model, which becomes a linear least square minimization problem, which can be computationally intensive if solving for each pixel. The authors were able to greatly reduce the complexity of the calculation using linear algebra representation readily solved using MATLAB.

These individual pixel noise characteristics are then incorporated into the noise reduction and smoothing filters in the image segmentation step, and then again in the single-particle localization using a maximum likelihood estimate (MLE) method. With the MLE calculation, the authors used an analytical approximation of the MLE likelihood function, which resembles a Poisson distribution, using normalized parameters. Therefore, they were able to directly implement the MLE using an established MLE method for either single or multi-emitter fitting analysis where a Poisson noise model is expected instead of having to create a new solution. The algorithm is implemented in MATLAB, and can be freely downloaded from the Nature Methods website.

WHO NEEDS TO USE THIS CORRECTION and HOW CAN IT BE IMPLEMENTED

SMSN demands attention to detail because of the level of detail in the question: Can I localize a single molecule that emits 100-1000 photons to within 10nm when each pixel optically represents 100nm? For most biologists, even those performing quantitative imaging, the questions are not addressed at a single pixel scale so applying pixel-by-pixel correction to account for small difference in gain and offset is not beneficial, and therefore not necessary.

References

  1. Huang, F. et al. Video-rate nanoscopy using sCMOS camera-specific single-molecule localization algorithms. Nat. Methods 10, 653–658 (2013).
  2. Robbins, M. S. & Hadwen, B. J. The noise performance of electron multiplying charge-coupled devices. Ieee Trans. 50, 1227 – 1232 (2003).
  3. Chao, J., Ward, E. S. & Ober, R. J. Fisher information for EMCCD imaging with application to single molecule microscopy. Conf. Rec. Asilomar Conf. Signals Syst. Comput. Asilomar Conf. Signals Syst. Comput. 1085–1089 (2010). doi:10.1109/ACSSC.2010.5757570
  4. Mortensen, K. I., Churchman, L. S., Spudich, J. A. & Flyvbjerg, H. Optimized localization-analysis for single-molecule tracking and super-resolution microscopy. Nat. Methods 7, 377–381 (2010).
  5. Pertsinidis, A., Zhang, Y. & Chu, S. Subnanometre single-molecule localization, registration and distance measurements. Nature 466, 647–651 (2010).
  6. Long, F., Zeng, S. & Huang, Z.-L. Localization-based super-resolution microscopy with an sCMOS camera Part II: Experimental methodology for comparing sCMOS with EMCCD cameras. Opt. Express 20, 17741 (2012).
Inquiry about Life Science Cameras

Contact us

 
Scroll to Top