Colour filter arrays, debayering and aliasing

A photosite counts photons but cannot tell their colour. To record colour images, almost every cinema camera places a mosaic of coloured filters over its sensor, so that each photosite records only red, green or blue light, and then reconstructs full colour for every pixel in processing. This reconstruction, called demosaicing or debayering, has a large effect on resolution, sharpness, colour accuracy and artefacts. Closely related is the problem of aliasing: when a sensor samples detail finer than it can resolve, false patterns and colours appear. This lesson explains colour filter arrays, how debayering works and what it costs in resolution, alternative approaches to colour capture, and the sampling theory behind aliasing, moiré and the optical low-pass filter.

The Bayer pattern

In 1976, Bryce Bayer at Eastman Kodak patented the colour filter array that bears his name and is still used in most cameras. The Bayer pattern arranges filters in repeating 2 × 2 blocks of one red, one blue and two green filters. Green gets twice as many photosites because the human visual system is most sensitive to green light and derives most of its perception of detail and brightness from the green and yellow parts of the spectrum. Each photosite therefore records only one colour channel, and the full-resolution image has two-thirds of its colour values missing, which must be estimated from neighbouring photosites.

Debayering

Debayering algorithms estimate the two missing colour values at every photosite from the values around it. The simplest method, bilinear interpolation, averages the nearest photosites of each colour; it is fast but blurs edges and produces coloured fringes along sharp boundaries. Better algorithms detect edges and interpolate along them rather than across them, use the correlation between colour channels (since the fine detail in red, green and blue usually coincides), and suppress false colour. High-quality debayering is computationally demanding, which is one reason raw footage can be debayered in post-production with slower, better algorithms than a camera can run in real time.

Because each photosite records only one colour, a Bayer sensor cannot resolve as much detail as its photosite count suggests. In practice, a well-debayered Bayer image resolves roughly 70 to 80 % of its photosite count in linear resolution, depending on the algorithm, the scene and the optical low-pass filter. This is a major reason why cinema cameras oversample: a sensor with about 6K or more photosites across, debayered and scaled down to a 4K delivery, produces a 4K image with genuinely 4K detail and fewer artefacts than a 4K Bayer sensor alone.

Other colour capture methods

Several alternatives to the Bayer pattern exist:

• Three-chip cameras use a prism to split light into red, green and blue images on three separate sensors, capturing all three colours at every pixel. They are common in broadcast cameras with 2/3-inch sensors but are bulky, require lenses designed for the prism, and are impractical for large sensors.
• Alternative mosaic patterns, such as Fujifilm's X-Trans with its 6 × 6 pattern, reduce the regularity that causes moiré, and some sensors add white (unfiltered) photosites for sensitivity.
• Pixel binning sensors group neighbouring photosites of the same colour into larger effective pixels, common in smartphones and some cameras' lower-resolution modes.
• Layered sensors, such as the Foveon design, stack photodiodes at different depths in the silicon, using the fact that different wavelengths penetrate silicon to different depths, but have struggled with noise and colour separation.
• Monochrome sensors with no colour filter capture full resolution and about twice the light, valued for black-and-white work.

For cinema, the single large Bayer sensor with high-quality debayering has proved the best balance of image quality, lens compatibility, size and cost.

Colour filters and colour science

The colour filters themselves are not perfect red, green and blue: each passes a range of wavelengths, and their spectral response curves overlap. The shape of these curves, together with the infrared-cut filter and the sensor's silicon response, determines how the camera responds to different colours and how accurately it can be transformed to a standard colour space. Filters with more overlap give higher sensitivity but require stronger colour correction, which amplifies noise; filters with less overlap give purer colour separation but less light. Manufacturers make different trade-offs, which is part of why cameras differ in their rendering of skin tones, foliage, saturated lights and mixed lighting, and why colour science, the processing that turns sensor data into accurate or pleasing colour, is a competitive feature (CINE-310).

Sampling and aliasing

A sensor samples a continuous image at discrete points. Sampling theory, associated with Harry Nyquist and Claude Shannon, states that to represent a signal faithfully, the sampling rate must be more than twice the highest frequency present. For images, frequency means spatial frequency: how many light-and-dark line pairs occur per millimetre or across the frame. A sensor with a given photosite pitch can faithfully represent detail only up to its Nyquist limit, one line pair per two photosites. Detail finer than this does not simply disappear; it is folded back into false, coarser patterns called aliases.

For a sensor with a 5 µm photosite pitch (0.005 mm), the Nyquist limit is 1 / (2 × 0.005) = 100 line pairs per millimetre. A Bayer sensor's red and blue channels are sampled at half that pitch in each direction, so their Nyquist limit is lower, which is why aliasing on Bayer sensors often appears as false colour.

Moiré and false colour

Aliasing appears in two common forms. Moiré is a pattern of coarse bands or swirls that appears when fine repeating detail, such as fabric weaves, brick walls, window screens, distant fences or LED video walls, interferes with the regular grid of photosites. False colour appears as coloured fringes or patterns on fine black-and-white detail, because the red, green and blue channels sample the detail at different positions. Both artefacts are especially visible in motion, where they shimmer and crawl as the camera or subject moves, and both are difficult to remove in post-production because the original detail has been lost.

The optical low-pass filter

Most cinema cameras place an optical low-pass filter (OLPF) in front of the sensor to reduce aliasing. Typically made of birefringent crystal plates, it splits each point of light into several slightly displaced points, blurring detail finer than the Nyquist limit before it reaches the photosites. The OLPF trades a small loss of sharpness for freedom from moiré and false colour. Its strength is a design choice: a strong OLPF gives clean, artefact-free images that may look slightly soft; a weak or absent OLPF gives crisper images with more risk of moiré. Some cameras offer interchangeable OLPFs optimized for different uses, such as one tuned for infrared-heavy lighting or for specific sensor modes, and the OLPF stack's thickness affects how lenses designed for different cameras perform, especially wide-angle lenses with steep ray angles.

Practical aliasing control on set

Even with a good OLPF, some subjects produce moiré, and camera teams learn to spot and avoid it. Practical steps include checking fine patterns such as costumes, screens and architectural details on a large, sharp monitor at full resolution during camera tests and blocking; changing the camera distance or focal length so that the pattern's spatial frequency moves away from the problem range; slightly softening focus or adding a diffusion filter; asking costume departments to avoid fine stripes and herringbone for key characters; and taking care when filming LED walls and computer screens, whose pixel grids interact with the sensor grid. Moiré is often invisible on small on-camera monitors, so it may only be discovered in editing unless checked deliberately.

Debayering in post-production

When footage is recorded raw, debayering happens in post-production software, and the choice of algorithm and settings affects the final image. Colourists and DITs can choose debayer quality, often trading speed for quality: a fast, lower-quality debayer for dailies and editing, and the highest quality for the final grade. Some raw formats allow debayering at reduced resolution for faster playback, such as half or quarter resolution, which is useful for viewing but should never be used for final output. Visual effects facilities may apply their own debayering to get the cleanest possible edges for keying and tracking.

LED walls and moiré

LED video walls used in virtual production and as backgrounds present a special moiré challenge. The wall's LED pixels form a regular grid that can interfere with the sensor's photosite grid, producing strong moiré. The risk depends on the wall's pixel pitch, the distance from the camera, the focal length and the focus. Practical guidance includes keeping the wall slightly out of focus, choosing walls with fine pixel pitch, keeping a minimum distance between camera and wall, and testing the specific combination of camera, lens and wall before production. Virtual production crews often calculate the risky distances and focal lengths in advance.

Colour accuracy and metamerism

Because camera filters do not match the human eye's colour sensitivities exactly, some pairs of colours that look identical to a person look different to a camera, and vice versa, a phenomenon called metamerism. It shows up when fabrics, paints or skin look slightly different on camera than in person, and especially under lighting with unusual spectra, such as some LED and fluorescent sources. Lighting with a full, continuous spectrum, measured by indexes such as the Television Lighting Consistency Index (TLCI) or the TM-30 metrics, reduces these surprises. Costume and art departments benefit from camera tests that reveal colours that render unexpectedly.

Sharpness, aliasing and the look

Decisions about aliasing are also aesthetic. Some cinematographers prefer the slightly softer, smoother rendering of a strong OLPF and an oversampled image, which resembles film; others prefer the crispness of a weaker filter for documentary or commercial work. Diffusion filters, vintage lenses and grain added in post can all shape the final texture. Whatever the choice, it should be tested and made deliberately, with attention to how fine patterns in costumes and sets will render in motion.