When White Text Disappears
May 14, 2026 · blog
I've got the grouping and timing working well. But now I'm staring at a different problem: my captions are invisible on half the frames.
White text, bottom of frame, black outline. Standard subtitle styling. Works perfectly on dark backgrounds. Completely unreadable on a bright sky, sandy beach, or white wall.
First Attempt: Hardcoded Thresholds
My first idea was simple: probe the bottom of the frame. If it's bright, add a dark background box behind the text.
I set a threshold at luminance 180 (out of 255). Anything above that = "bright" = add background.
Worked on my test videos. Failed immediately on a moody low-light video where the "bright" regions were actually at luminance 90. And on an outdoor vlog where the average was 200, meaning almost everything triggered the background — even scenes where white text was perfectly fine.
A single threshold can't work across videos with fundamentally different lighting.
The Insight: Adaptive Thresholds
What's "bright" depends on the video. A luminance of 140 is bright in a dark film. It's medium in an outdoor vlog. I need per-video thresholds.
I compute them from the video's own distribution:
mean = average luminance of all bottom-region samples
stddev = standard deviation of those samples
bright_threshold = clamp(mean + stddev, [140, 220])
dark_threshold = clamp(mean - stddev, [30, 100])
The clamps prevent absurd values (a completely black video shouldn't set "bright" at luminance 20). But within those bounds, each video defines its own normal.
Probing the Frame
For each caption's timestamp, I extract luminance from two regions using FFmpeg:
- Bottom 20% of the frame (where captions normally sit)
- Top 20% of the frame (escape route if bottom is too bright)
The FFmpeg command:
ffmpeg -ss <timestamp> -i video.mp4 -vf "crop=iw:ih*0.2:0:ih*0.8,format=gray" -f null -
This gives me a grayscale mean (0–255) for that region at that moment. I deduplicate timestamps within 500ms windows so I'm not probing the same frame twice.
The Decision Tree
Once I have luminance values for top and bottom, the logic is:
| Bottom | Top | What I do |
|---|---|---|
| Bright | Dark | Move captions to the top — it's readable there |
| Bright | Not dark | Keep bottom, add a 60% black background box |
| Medium | Any | White text + 30% semi-transparent background |
| Dark | Any | White text, no background — it's already readable |
The best case is bottom-dark: no background needed, text looks clean. The worst case is both regions bright (outdoor noon shot) — I keep bottom and add a strong background. It's not beautiful but it's readable.
Performance Concern
Each probe is an FFmpeg call. 80 captions = 80 FFmpeg invocations? That's slow.
The deduplication helps — captions within 500ms of each other share one probe. In practice a 4-minute video needs ~40 unique probes. Each takes about 0.5s. Total: ~20 seconds.
Not instant, but acceptable. And it only runs once — results get cached with the rest of the pipeline state.
Results
Night-and-day difference. Videos with beach scenes, bright windows, white backgrounds — captions are now always readable. Either they move to the top where it's dark, or they get a tasteful background.
I'm starting to feel like all these pieces (timing, grouping, rendering, positioning) should live in one tool. Right now they're scattered scripts. But that's a problem for later — for now, each piece works on its own.