Image sensors and pixels

At the heart of every digital camera is a sensor: a grid of millions of tiny light-sensitive elements, called pixels or photosites, built on a silicon chip. The sensor determines how much light the camera can collect, how finely it can record detail, how cleanly it records shadows and how well it preserves highlights. This lesson explains how a photosite works, the measures that describe sensors, sensor sizes and why total light matters more than pixel size, how resolution relates to megapixels and print size, and the design features, such as back-side illumination and stacked construction, that distinguish modern sensors. CINE-201 covers sensor physics for motion pictures in more depth; here the emphasis is on what matters for still photography.

How a photosite works

Each photosite contains a photodiode, a region of silicon that collects electrons freed when photons of light strike it, by the photoelectric effect. During the exposure, electrons accumulate in proportion to the light received: twice as much light gives twice as many electrons. At the end of the exposure, the camera measures the charge in each photosite, amplifies it, and converts it to a number with an analogue-to-digital converter (ADC). The numbers form the raw image. A tiny lens, a microlens, sits above each photosite to funnel light into the photodiode, and a colour filter above it lets only red, green or blue light through (Lesson 4).

Three properties describe how well a photosite works. Quantum efficiency is the fraction of incoming photons that are converted into electrons; modern sensors convert well over half. Full-well capacity is the maximum number of electrons a photosite can hold before it saturates, at which point it records pure white and any further detail is lost. Read noise is the random variation added each time the charge is measured, a few electrons in good sensors. Together, full-well capacity and read noise set the sensor's dynamic range (Lesson 3).

Light is grainy: shot noise

Light arrives as discrete photons, randomly spaced in time. Even under perfectly steady illumination, two identical photosites exposed for the same time will collect slightly different numbers of photons. This variation, called photon shot noise, is a property of light itself, not of the sensor. Its size is the square root of the average number of photons: a photosite collecting an average of 10,000 photons varies by about 100 (1 %), while one collecting 100 photons varies by about 10 (10 %). This is why images made with little light look noisy regardless of the camera, and why collecting more light is the most effective way to reduce noise.

Quadrupling the light doubles the signal-to-noise ratio. More light can come from a wider aperture, a longer shutter speed, a brighter scene, or a larger sensor that collects light over a larger area.

Sensor size and total light

For a given exposure, meaning the same scene brightness, f-number and shutter speed, every square millimetre of sensor receives the same amount of light. A larger sensor therefore collects more light in total, in proportion to its area. A full-frame sensor (36 × 24 mm, 864 mm²) has about 2.3 times the area of an APS-C sensor (about 23.5 × 15.6 mm, 367 mm²) and about 3.8 times the area of a Micro Four Thirds sensor (17.3 × 13 mm, 225 mm²). When images from each are viewed at the same size, the full-frame image is built from more total light and shows less noise, by roughly one stop compared with APS-C and two stops compared with Micro Four Thirds, at the same exposure settings.

This comparison assumes the same f-number. As Lesson 6 explains, a smaller format can use a wider aperture to collect the same total light with the same depth of field, which is the idea of equivalence. Smartphones compensate for their small sensors by combining many frames, effectively collecting light over a longer time (Lesson 10).

Pixel size: a common misunderstanding

A popular belief holds that larger pixels always produce cleaner images, so a 24-megapixel full-frame camera must be less noisy than a 61-megapixel one. At the level of individual pixels viewed at 100 %, the larger pixel does collect more light and looks cleaner. But images are viewed or printed at a fixed size, not pixel by pixel. When the 61-megapixel image is shown at the same size as the 24-megapixel image, several of its small pixels contribute to each area of the final image, and their noise averages out. Because both sensors collect the same total light, noise in the final image is very similar, provided the sensor technology is comparable. Small pixels carry a slight penalty from read noise in the deepest shadows, and very small pixels reach the diffraction limit sooner (Lesson 5), but for most photography, sensor size and technology matter far more than pixel count.

Resolution, megapixels and print size

Megapixels describe the number of pixels, and therefore the maximum amount of detail a sensor can record. Whether that detail is visible depends on the lens, focus, camera shake and how the image is displayed. For prints, a common guideline is about 300 pixels per inch (ppi), about 118 pixels per centimetre, for close viewing, while large prints viewed from a distance look sharp at much lower densities. A 24-megapixel image (6,000 × 4,000 pixels) prints at 300 ppi to 20 × 13.3 inches (about 51 × 34 cm); a 61-megapixel image (9,504 × 6,336 pixels) to about 32 × 21 inches (80 × 54 cm). Resolution also matters for cropping: a high-resolution file allows a wildlife photographer to crop heavily and still print large. Doubling linear resolution requires four times the pixels, which is why the visible difference between 24 and 33 megapixels is modest.

Readout, bit depth and processing

After exposure, the sensor's charges are read out row by row and digitized, typically at 12 or 14 bits per pixel in raw files (4,096 or 16,384 levels). The speed of readout matters for electronic shutters and burst shooting (Lesson 8). The analogue gain applied before digitizing changes with the ISO setting: at higher ISO, the signal is amplified more, which can reduce the effect of noise added later in the chain, though it also reduces the maximum signal that can be recorded before clipping (Lesson 3). The processor then corrects for defective pixels and sensor quirks before the image is processed or saved raw.

Sensor technologies

Several developments have improved sensors in recent years. Back-side illuminated (BSI) sensors place the wiring behind the photodiodes rather than in front, so more light reaches them, a particular benefit for small pixels. Stacked sensors bond the light-sensitive layer to a separate layer of fast readout circuitry and memory, greatly increasing readout speed, which reduces electronic shutter distortion and allows very fast bursts and blackout-free viewfinders. Dual-gain designs switch photosites between two conversion gains for cleaner shadows at higher ISO. Global shutter sensors expose all pixels simultaneously (Lesson 8). Each technology has trade-offs in cost, heat and, sometimes, dynamic range.

Cleaning and caring for the sensor

Dust on the sensor, strictly on the filter stack in front of it, appears as dark, soft spots in images, most visible at small apertures against plain areas such as skies. Most cameras shake dust off ultrasonically when switched on, and many mirrorless cameras close the shutter when changing lenses to protect the sensor. To check for dust, photograph a plain bright surface at f/16 and inspect the image. Loose dust can be removed with a hand blower; stubborn particles need sensor swabs and cleaning fluid used carefully, or professional cleaning. Never use compressed air cans, which can spray propellant, and avoid changing lenses in dusty or windy conditions where possible.

From photons to numbers: a worked chain

It helps to follow a single photosite through an exposure. Suppose a photosite on a full-frame sensor receives 20,000 photons during an exposure. With a quantum efficiency of 50 %, about 10,000 of them free electrons, which are stored in the photosite's well. Shot noise is the square root of the signal, about 100 electrons, so the signal-to-noise ratio from light alone is about 100 to 1. When the exposure ends, the charge is converted to a voltage, amplified according to the ISO setting, and digitized by an analogue-to-digital converter into a number, for example between 0 and 16,383 for a 14-bit converter. Read noise added by the electronics, perhaps 3 electrons at base ISO, is tiny compared with the shot noise in this well-exposed photosite, but it becomes significant in deep shadows where only a few dozen electrons are collected.

Microlenses and fill factor

Not every part of a photosite's surface is sensitive to light; wiring and transistors occupy some of the area, and the fraction that is sensitive is called the fill factor. To recover the light that would otherwise fall on insensitive areas, sensors have an array of tiny microlenses, one over each photosite, that focus incoming light onto the light-sensitive region. Back-side illuminated (BSI) sensors move the wiring behind the photodiode, raising the effective fill factor, which particularly benefits small photosites. Microlenses work best when light arrives nearly perpendicular to the sensor; light striking the edges of the sensor at steep angles, common with wide-angle lenses designed for film or rangefinder cameras, can cause colour shifts and darkening in the corners. Manufacturers shift the microlenses progressively toward the edges to compensate, and some sensors designed for short-flange rangefinder lenses use special microlens arrangements.

Sensor heat, long exposures and dark current

Sensors generate a small signal even in complete darkness, called dark current, caused by thermal energy freeing electrons in the silicon. Dark current roughly doubles for every 6 to 8 °C rise in temperature, so it matters most in long exposures, hot weather and video recording, when the sensor warms up. It appears as an increase in noise and as hot pixels, individual photosites that are much brighter than their neighbours. Long-exposure noise reduction, offered by most cameras, takes a second exposure of the same length with the shutter closed, called a dark frame, and subtracts it from the image, removing hot pixels and fixed patterns at the cost of doubling the time per shot. Astrophotographers often take dark frames separately at the same temperature and subtract them during processing, so that the camera is free to continue shooting. Cameras with sensors cooled by thermoelectric devices are used for scientific and astronomical imaging.

Stacked sensors and readout speed

Readout speed has become one of the most important sensor characteristics. Conventional sensors read out row by row through circuitry at the sensor's edges, taking perhaps 1/15 to 1/30 s to read a full frame. Stacked sensors place memory and processing circuitry on separate layers bonded directly behind the photodiode layer, allowing many rows to be read in parallel and data to be buffered at very high speed. Readout times of 1/120 s to 1/250 s or faster allow electronic shutters with minimal distortion, blackout-free viewfinders, burst rates of 20 to 120 frames per second, and faster autofocus calculations, since the camera can analyse many more frames per second. The trade-off historically has been higher cost and sometimes slightly lower dynamic range, though this gap has narrowed. Partially stacked designs offer some of the speed benefits at lower cost.