Audio-Video Sync Drift: Diagnosing Lip-Sync Problems That Grow Worse Over Time
Why audio and video drift apart in long streams — sample-rate mismatch, PTS handling, muxing errors — and the measurement technique that finds the source.
Lip-sync drift is the bug that looks like magic: the stream starts fine, and five minutes in, dialogue is a half-second late. Then worse. The cause is almost never random — it’s a systematic clock error, and it has exactly three common origins.
The Three Origins of Sync Drift
| Cause | Symptom | Typical Error Rate |
|---|---|---|
| Sample-rate mismatch (44.1k vs 48k in capture) | Linear drift — same rate everywhere | ~43ms per minute |
| Muxing offset | Constant offset, doesn’t grow | Fixed ±30–80ms |
| VFR timestamp mangling | Drift that jumps at edits | Irregular |
Sample-Rate Mismatch — The Silent Killer
A capture card that reports 48kHz but is actually clocked at 44.1kHz creates a ~43ms-per-minute drift. In five minutes you’re nearly a quarter-second behind — viewers notice around 80–100ms. The diagnostic: compare the audio stream’s declared rate against its actual frame count. ffprobe shows declared rate; the real rate is (audio_samples / duration). If they don’t match, resample at ingest.
# Resample to the true rate and preserve duration
ffmpeg -i input.mp4 -af "aresample=48000" -c:v copy output.mp4
PTS vs. DTS — Why It Goes Sideways in Muxing
Every audio and video packet carries a presentation timestamp (PTS). Muxers that write timestamps in different timebases, or tools that rewrite PTS without fixing up the other track, produce a constant offset — the stream starts wrong and stays wrong. Fix with -itsoffset or remux cleanly:
# Shift audio track by -80ms (audio early) — measure first
ffmpeg -i input.mp4 -itsoffset 0.08 -i input.mp4 -map 0:v -map 1:a -c copy synced.mp4
How to Measure It
Don’t eyeball it. Use a clapper test signal — a loud click with a visible flash at a known time:
- Record a signal generator video (tone burst + frame flash at 1s intervals).
- Play it through your full pipeline.
- Compare the flash frame to the audio peak frame — the delta is your offset.
If the delta grows each iteration, it’s sample-rate drift. If it’s constant, it’s a muxing offset.
“Every lip-sync bug we’ve diagnosed came down to one of three things: a clock that wasn’t the clock it claimed to be, a muxer that didn’t carry PTS correctly, or a player that guessed the audio clock. None of them are subtle once you measure them.”
The full measurement rig, expected tolerances (±40ms broadcast, ±80ms web), and the FFmpeg test-signal recipe are in the AV sync drift diagnostic playbook.