Recording, timecode and sync workflows

Film is recorded on two systems: the camera captures picture, and a separate recorder captures sound. This dual-system approach gives the sound team better microphones, more tracks and better preamplifiers than any camera, but it means picture and sound must be brought back together precisely. A sync error of just two frames, about 80 milliseconds, is visible to many viewers on a close-up. This lesson explains timecode, frame rates, slates and metadata, and follows the files from the recorder to the dialogue editor's session. FILM-220 lesson 9 introduced sync from the picture editor's side; here we look at it from the sound department's side.

Why double-system sound needs sync

Cameras and recorders each contain their own clock. Even two high-quality devices drift slightly apart over time because their crystal oscillators run at very slightly different speeds. Over a short take the drift is negligible, but over a long take, such as a 20-minute documentary interview or a concert, it can add up to frames. Sync therefore needs two things: a way to line up the start of each take, and a way to keep the devices running at the same speed during the take.

The clapperboard, or slate, solves the first problem. When the clapper sticks close, the frame in which they meet and the sharp transient in the audio mark the same instant. Editors line up the visible closing with the audio spike and the take is in sync. Waveform-matching software can also line up a recorder's audio with the camera's own scratch audio automatically. Timecode solves both problems more completely by writing the same running clock into both picture and sound files.

Timecode basics

SMPTE timecode, standardized by the Society of Motion Picture and Television Engineers, labels every frame with hours, minutes, seconds and frames, written HH:MM:SS:FF. On set, the sound recorder is usually the master timecode generator, and small timecode boxes jam-synced to it are attached to each camera. Jam sync means setting each device's clock to match the master; good timecode boxes then hold accurate time for many hours, with drift of well under a frame per day. Most productions use time-of-day timecode, in which the clock matches the real time of day, so files from different cameras and recorders can be matched easily.

Timecode must run at the project's frame rate, and every device must agree. Mismatched timecode rates are among the most common and painful sync problems in post. The mixer confirms the frame rate with the camera department and the post-production supervisor before the shoot and writes it on the sound report.

The 0.1 per cent problem: 23.976, 29.97 and drop-frame

North American television originally ran at 30 frames per second, but when colour broadcasting was introduced in the 1950s the rate was reduced by a factor of 1000/1001 to about 29.97 fps to avoid interference. Film shot at 24 fps and transferred to video was likewise slowed to 23.976 fps. These fractional rates persist in digital production. A timecode counter that counts 30 frames per second while the video actually runs at 29.97 will slowly fall behind the real clock, by about 3.6 seconds per hour.

Drop-frame timecode solves this for 29.97 by skipping frame numbers, not actual frames: it skips frame numbers 00 and 01 at the start of every minute except every tenth minute, so the timecode stays within a fraction of a second of real time. Drop-frame timecode is usually written with a semicolon before the frames, such as 01:00:00;00. There is no standard drop-frame timecode for 23.976, so 23.976 projects use non-drop timecode that runs slightly slower than the wall clock.

The sound department meets the 0.1 per cent problem in a specific way. If audio is recorded at 48 kHz against 24 fps timecode but the picture is shot at 23.976 fps, the two will drift apart by 0.1 per cent: about 3.6 seconds over an hour, or nearly 3 frames in two minutes. Older workflows recorded audio at 48.048 kHz to compensate; today the simplest rule is that the recorder's timecode must match the camera's frame rate exactly, and the sample rate stays at 48 kHz.

for minutes in [1, 2, 10, 60]:
    t = minutes * 60
    drift_s = t * (1 - 1000 / 1001)
    print(f'{minutes:3d} min: drift {drift_s * 1000:7.1f} ms = {drift_s * 23.976:5.2f} frames at 23.976 fps')
for fps in [23.976, 24, 25, 29.97]:
    print(f'{fps:6.3f} fps: 1 frame = {1000 / fps:5.2f} ms; 2 frames = {2000 / fps:5.1f} ms')
print('29.97 non-drop clock error per hour:', round(3600 * (1 - 1000 / 1001) * 1000 / 1000, 2), 's')

  1 min: drift    59.9 ms =  1.44 frames at 23.976 fps
  2 min: drift   119.9 ms =  2.87 frames at 23.976 fps
 10 min: drift   599.4 ms = 14.37 frames at 23.976 fps
 60 min: drift  3596.4 ms = 86.23 frames at 23.976 fps
23.976 fps: 1 frame = 41.71 ms; 2 frames =  83.4 ms
24.000 fps: 1 frame = 41.67 ms; 2 frames =  83.3 ms
25.000 fps: 1 frame = 40.00 ms; 2 frames =  80.0 ms
29.970 fps: 1 frame = 33.37 ms; 2 frames =  66.7 ms
29.97 non-drop clock error per hour: 3.6 s

How much sync error is acceptable

Research on audiovisual synchrony, and broadcast recommendations such as ITU-R BT.1359, show that viewers are more tolerant of sound arriving late than early, because in nature light reaches us before sound. Broadcast guidance commonly treats sound leading picture by more than about 45 ms or lagging by more than about 125 ms as detectable, with tighter targets for production. In film post-production the standard is frame accuracy: sound editors work with sub-frame precision and aim for zero offset on dialogue. Sound effects may be slipped by a frame or two for creative reasons, such as making a punch land on the impact frame rather than the frame before.

Slates, smart slates and good habits

A smart slate displays running timecode received wirelessly from the recorder, so the camera sees the timecode number at the clap. Even with timecode on every device, mixers ask for a visible clap on each take, because it provides a backup and allows quick visual checks. Good slating habits include holding the slate in focus and in frame, clapping clearly but not explosively near actors, using a soft "tail slate" (clapped upside down at the end) when the head slate was missed, and calling "second sticks" when the first clap was missed.

When multiple cameras roll, each camera should see the slate or have its own timecode box. For documentary, where clapping may be intrusive, timecode boxes on every device and a quiet hand clap at a natural break serve the same purpose.

Metadata and file naming

Broadcast WAV files carry metadata in their headers: scene, take, tape or roll name, track names, timecode start, frame rate, sample rate and notes. This metadata travels with the files to editorial and later to the dialogue editor. When it is accurate, the sound team can automatically match each production file to the picture edit and find alternative takes in seconds. When it is wrong or missing, assistants spend days searching.

Mixers enter metadata on the recorder or a connected tablet as the day goes: scene and take numbers that match the script supervisor's notes, track names such as BOOM, MARA LAV and DEV LAV rather than "track 3", and circle takes the director liked. The sound report lists the same information with comments on problems and wild lines.

From the set to editorial

At the end of each day, sound files are delivered to editorial, usually by copying from the recorder's cards to drives or uploading through the production's data workflow. The digital imaging technician or assistant editor syncs sound to picture using timecode and checks the result, creating synced dailies for the director and producers. On many productions the sync is done in the dailies software, then imported into the picture editing system with the production audio attached. The picture editor works mostly with the mix track, but the isos travel along, hidden, for later use.

Cloud and camera-to-cloud workflows now send proxy media and production audio from set to editorial within minutes. For sound, the key rule is unchanged: the original files, with full metadata and all tracks, must reach post-production untouched. Compressed or mixed-down versions are for review only.

Turnover: from picture edit to sound edit

At turnover, the assistant editor exports the edit's audio as an AAF (or the older OMF) file. The AAF contains the timeline's audio clips with their positions and, importantly, handles: extra seconds of audio beyond each cut, usually 5 to 10 seconds, so that dialogue editors can extend lines, find room tone and reach alternative microphones. The export also includes a reference video with burned-in timecode and a guide track, and an edit decision list that lets the sound team find the original source files.

The dialogue editor then performs a conform, sometimes called a re-link: software reads the AAF and edit decision list, finds the original multitrack production files using the metadata, and rebuilds the editor's timeline with all isolated tracks available. A successful conform depends on everything that happened on set: matching timecode, consistent file names, correct metadata and complete media.

Picture changes and conforming

In reality, picture is rarely truly locked. When the director makes changes after turnover, the sound team must move every track to match the new cut. Change lists produced by the editing system show which shots moved, were trimmed or were added. Conform tools can apply these changes automatically, but large changes still require manual checking of every edit. This is why producers should treat the picture lock as a firm date, and why sound teams charge for conforming when changes are made after lock.