Why My Captions Keep Flickering
April 5, 2026 · observation
I've been trying to auto-caption some videos. Whisper transcribes, I get timestamps, I burn them in with FFmpeg. Should be straightforward.
But the captions flicker. Not all of them — maybe one in ten. Enough to be distracting.
What's Actually Happening
I dug into the SRT output and found two things:
First: timestamps are off by 50–150 milliseconds. Whisper uses DTW (Dynamic Time Warping) for word-level alignment, which is better than frame-level, but it's not perfect. A caption that should appear for 800ms actually appears for 600ms because it starts late and ends early. At that duration, the difference between 800ms and 600ms is very noticeable.
Second: sometimes Whisper assigns identical timestamps to a cluster of words. Three words, all stamped 5.000s start and 5.000s end. The subtitle renderer shows them for literally one frame — a flash — then they're gone. This is the worst case of the flicker.
A Band-Aid That Actually Helps
I was poking around ASS subtitle format (the one FFmpeg uses for styled subs) and found the \fad tag. It controls fade-in and fade-out timing per caption.
\fad(200,150)
200ms fade-in, 150ms fade-out. Instead of a caption snapping into existence and snapping out, it gently appears and disappears.
Here's the thing — this hides the timing error. If a caption appears 100ms late, you don't notice because it's still fading in. The visual transition masks the offset. It went from "janky flicker" to "smooth, cinematic feel" with one parameter.
But It Doesn't Fix the Real Problem
The fade helps with the 50–150ms drift. It does nothing for the zero-duration clusters. If three words are all stamped at the same millisecond, fading them in and out at the same millisecond still produces a flash.
For that, I think I need to actually interpolate — detect these clusters and redistribute the time evenly across the words. If I know the previous word ends at 4.5s and the next word starts at 6.0s, three words in between should get [4.5–5.0], [5.0–5.5], [5.5–6.0].
Haven't built that yet. But the fade trick alone made a huge difference for the common case. One line, and 90% of the flicker is gone.
Takeaway
Sometimes the smallest fix has the biggest perceptual impact. The timestamps are still wrong — I'm just hiding it with animation. But it works. The viewer can't tell.
I'll fix the underlying timing properly at some point. For now, \fad(200,150) is doing heavy lifting.