Visual effects in the production pipeline

Visual effects are now part of almost every film and series, not only spectacles with dragons and spaceships. A period drama removes modern signs and adds a horse-drawn street; a contemporary thriller replaces the view outside a car window; a comedy erases a boom shadow; a documentary stabilizes and cleans an archival shot. Industry surveys regularly find that the large majority of shots in a major studio feature involve some digital work, and even modest independent films contain dozens of invisible fixes. For a cinematographer, this means that a great deal of what is photographed will be combined, extended, corrected or replaced after the camera stops rolling. This course is about shooting for that future: planning shots with the visual effects team, photographing plates and elements correctly, capturing the data that artists need, lighting green screens and LED volumes, and keeping the image consistent from set to final delivery. This first lesson places visual effects within the production pipeline and defines the roles and vocabulary used throughout the course.

Special effects and visual effects

The industry distinguishes between special effects (SFX), which happen physically on set in front of the camera, and visual effects (VFX), which are created or altered in post-production. Special effects include rain, snow, wind, smoke, fire, breakaway props, squibs, pyrotechnics and mechanical rigs, and are managed by a special effects supervisor and crew who are licensed for explosives and pyrotechnics where required. Visual effects include compositing, digital environments, computer-generated characters and creatures, crowd multiplication, wire and rig removal, digital makeup, and the countless clean-ups that audiences never notice. The two disciplines work together: a practical explosion photographed on set may be enhanced with digital debris, and a digital creature may interact with practical water splashes. A cinematographer should know which effects are planned as practical, which as digital and which as a combination, because each choice changes how the shot must be lit and photographed.

A short history

Visual effects are as old as cinema. Georges Méliès used stop tricks, multiple exposures and painted sets in the 1890s and early 1900s. Matte paintings on glass, placed between the camera and the set, extended scenery from the silent era onward, and the Schüfftan process used mirrors to combine actors with miniatures in films such as Metropolis (1927). Optical printers, which rephotographed film elements, allowed traveling mattes and complex composites through the middle of the twentieth century, and blue screen photography developed alongside them. Rear projection placed actors in front of moving backgrounds, most visibly in car interiors, while front projection, used famously for the opening of 2001: A Space Odyssey (1968), offered brighter and sharper backgrounds using retroreflective screens. Motion control, developed for Star Wars (1977), made complex repeatable camera moves possible. Digital compositing and computer-generated imagery grew rapidly from the late 1980s, with landmarks such as Terminator 2 (1991) and Jurassic Park (1993), and digital intermediates and fully digital pipelines became standard in the 2000s. Since around 2019, LED volumes have revived the principle of rear projection with real-time rendered backgrounds, a technique known as in-camera visual effects. Understanding this lineage helps cinematographers see that many modern problems, such as matching light between foreground and background, are old problems with new tools.

The VFX pipeline

A visual effects shot passes through a pipeline of departments, each building on the work of the last. The exact pipeline varies by vendor, but a typical sequence is:

• Development and breakdown: the script is analyzed for VFX work, and shots are designed, counted and budgeted (Lesson 2).
• Previsualization: rough animated versions of complex sequences are created to plan shots, camera moves and technical requirements (Lesson 3).
• Production: plates and elements are photographed, and data is captured on set (Lessons 4 to 8).
• Turnover: editorial delivers selected takes, with frame ranges, metadata, reference and notes, to the VFX vendors.
• Matchmove and layout: the camera's motion is reconstructed in 3D, and CG elements are placed in the scene (Lesson 6).
• Asset creation: models, textures, rigs and shading for characters, props and environments.
• Animation and simulation: characters move; water, fire, smoke, cloth and destruction are simulated.
• Lighting and rendering: CG elements are lit to match the plate and rendered, often in many separate passes.
• Compositing: everything is combined with the plate, with roto, keying, paint and color matching to produce the final image.
• Review and delivery: shots are reviewed by the VFX supervisor, director and studio, revised and delivered back to editorial and the colorist (Lesson 12).

Each stage depends on accurate information from production. A badly exposed plate, missing lens data or an unexplained camera move can cost days of artists' time, multiplied across hundreds of shots.

Who does what

Several roles connect the camera department with visual effects:

• The VFX supervisor, hired by the production, is responsible for the creative and technical quality of all visual effects. On set, they advise the director and cinematographer on how to shoot each VFX shot and approve plates before the company moves on.
• The VFX producer manages budgets, schedules, vendor contracts and shot counts.
• The on-set VFX team, often called data wranglers or VFX coordinators, records camera and lens data, photographs references, captures HDRIs and surveys, and keeps the shot logs.
• Vendor VFX supervisors lead the artists at each visual effects company and may attend the shoot for their sequences.
• The previs and virtual production teams create previs, run real-time engines and operate LED volumes (Lessons 3, 10 and 11).
• The director of photography designs the photographic approach and owns the look of the plates, working with the VFX supervisor so that the photographed image and the final composite feel like one film.

Good collaboration begins in pre-production: the cinematographer and VFX supervisor should agree on formats, lenses, frame rates, color pipeline and the approach to every major VFX sequence long before the first plate is shot.

Vocabulary for the course

Visual effects work uses a specialized vocabulary. A plate is any photographed footage that will be used in a VFX shot, whether it contains actors (a foreground or principal plate) or not (a background or clean plate). An element is a separately photographed component, such as smoke, dust, sparks or a crowd section. A matte is an image that defines which parts of a frame are visible, used to combine layers; a key is a matte generated automatically from a color, such as a green screen. Roto, short for rotoscoping, is the manual drawing of mattes frame by frame. Paint is frame-by-frame cleanup of unwanted detail. A matchmove or camera track reconstructs the real camera's movement in 3D software. CG stands for computer-generated imagery. A comp is a composite, and a temp comp is a quick version used in editing before final effects are complete. Turnover is the delivery of plates from editorial to the VFX vendors, and a shot's handles are extra frames before and after the cut range.

How VFX changes the cinematographer's job

Visual effects add responsibilities to cinematography. The cinematographer must imagine the final image, not only what is in front of the lens: a green screen stage may become a canyon at sunset, so the actors must be lit as if by that sun. The cinematographer must think about the needs of later departments: exposure that preserves detail for keying, lenses and data that allow accurate matchmoving, frame rates and shutter angles that match between plates and elements, and consistent color pipelines. The cinematographer must also be prepared to accept constraints, such as fixed lenses for a motion control pass or limited camera positions in an LED volume, and to defend the photographic integrity of the film when visual effects threaten to make it feel weightless or artificial. Many cinematographers stay involved in post-production, reviewing VFX shots for lighting continuity and grading them alongside the photographed material.

Practical versus digital

Deciding whether to achieve an effect practically, digitally or in camera is a creative and financial question. Practical effects and real locations give actors something to respond to, produce natural light interaction and often look more convincing, but they can be expensive, dangerous or impossible to repeat. Digital effects offer unlimited flexibility and safety but may look artificial if poorly integrated, and their cost depends on the number of shots and the complexity of each. In-camera VFX with LED volumes give final images on set and realistic reflections but require large investment in content before shooting and limit some camera choices. Many productions combine approaches: a real car on a process trailer with an LED wall providing reflections and window views, or a partial set built to head height and extended digitally above. The cinematographer contributes to these decisions by explaining what each option means for lighting, lenses, schedules and the look of the film.

Course overview

Lesson 2 covers breakdowns, budgets and bidding; Lesson 3, previs, techvis and postvis; Lesson 4, plates and elements; Lesson 5, green and blue screen; Lesson 6, matchmoving and tracking markers; Lesson 7, on-set data capture; Lesson 8, motion control and repeatable moves; Lesson 9, set extensions, digital environments and digital doubles; Lessons 10 and 11, LED volumes and in-camera VFX; and Lesson 12, working with the VFX team, review and delivery, and the capstone project, a complete VFX shooting plan for a sequence.

Reading a script for effects

Even before a VFX supervisor is hired, a cinematographer can read a script with effects in mind. Mark every moment that could not be photographed safely, legally or affordably as written: a city that no longer exists, a crowd of thousands, a vehicle crash, a creature, weather on cue, a performer in two places at once. For each, ask three questions. Could it be done practically with special effects, stunts or art department builds? Could it be done partly in camera and finished digitally? Or must it be created almost entirely in the computer? Writing these first instincts down gives the director and producers an early picture of the effects load, and it starts the conversation about method long before budgets are fixed. The most useful habit is to think of every effects moment as a combination of real photography and digital work, then to ask which part each department does best.