Whisper Silence Hallucination

Why a transcriber writes words over empty audio, and the gate that stops it · 2026-09-30

Given silence, Whisper emits its favourite caption anyway. Running faster-whisper base over a 133.8 second track produced five words, each of them You, placed at 00:00, 00:30, 01:00, 01:30 and 02:00. The audio contained no sound at any point.

The evidence the audio was empty

$ ffmpeg -i clip.mp4 -af volumedetect -f null -

  n_samples:   2140160
  mean_volume: -91.0 dB
  max_volume:  -91.0 dB

Mean equal to max is the decisive reading. It means every one of the 2,140,160 samples holds the same value, so the track is digital silence from the first sample to the last. A 16 bit floor is around -96 dB, so -91.0 is the noise floor of an empty container.

Track	mean dBFS	max dBFS	Reading
Empty clip A	-91.0	-91.0	digital silence
Empty clip B	-91.0	-91.0	digital silence
Empty clip C	-91.0	-91.0	digital silence
Real speech A	-21.6	-0.4	has signal
Real speech B	-24.3	-0.0	has signal

The separation is absolute. There is no value between -24.3 and -91.0.

Why the model does it

1. Whisper processes audio in 30 second windows and must emit a token sequence for each one.

2. With no acoustic evidence, the decoder falls back on its language prior.

3. That prior came largely from YouTube caption tracks, where You, Thank you for watching and Thanks for watching! are among the most common closing lines in the corpus.

4. So each empty window is filled with the statistically likeliest caption text.

5. The timestamps prove the mechanism. The five tokens fall at exactly 00:00, 00:30, 01:00, 01:30 and 02:00. Perfectly regular 30 second spacing is a per-window artifact, never the rhythm of speech.

Why the first gate missed it

The check for success was words > 0. Five is greater than zero, so the run was stamped OK and five fabricated words were reported as content. That is counting output rather than measuring it. A count cannot tell invented text from transcribed text.

A duration based silence rule did not save it either. A rule that cuts at the first silent run lasting past five minutes cannot fire on a clip of 133.8 seconds, because a 133.8 second file structurally cannot contain a 300 second run.

The fix is to measure level before transcribing, rather than to judge the output afterwards.

def volume_stats(path):
    r = run(["ffmpeg", "-i", path, "-af", "volumedetect", "-f", "null", "-"])
    mean = float(re.search(r"mean_volume: ([-0-9.]+) dB", r.stderr).group(1))
    mx   = float(re.search(r"max_volume: ([-0-9.]+) dB",  r.stderr).group(1))
    dead = abs(mean - mx) < 0.01 or mx <= -60.0
    return mean, mx, dead

# called BEFORE the model is loaded
if dead_audio:
    return "NO_SPEECH", f"digital silence, mean {mean} dBFS and max {mx} dBFS"

Whisper is never started on an empty track, so there is no window in which it can invent a token. A second guard remains as a backstop: any output under 30 words carrying three or fewer unique tokens is labelled a hallucination rather than counted as speech.

Result after the fix

Clip	Before	After
Empty clip A, 0.37 min	EMPTY_AFTER_TRANSCRIBE	NO_SPEECH, basis printed
Empty clip B, 0.33 min	EMPTY_AFTER_TRANSCRIBE	NO_SPEECH, basis printed
Empty clip C, 2.23 min	OK, 5 words "You"	NO_SPEECH, basis printed
Real speech A, 11.47 min	OK	OK, unchanged
Real speech B, 4.89 min	OK	OK, unchanged

Correction: never truncate at silence

An earlier version of this page recommended cutting the transcription at the first silent run past five minutes. That was wrong, and it was corrected the same day by the person whose recordings these were.

Meetings pause. People step away for fifteen minutes and come back, and the material after the pause is often the part worth having. A transcriber that stops at the first long silence throws away the second half of the meeting.

The rule now: transcribe the whole file, end to end, every time.

Gap detection still runs, at silencedetect=noise=-40dB:d=900, to FIND breaks rather than to act on them.

Each gap of 15 minutes or longer is labelled in the header, for example 00:05 to 16:05 (16.0 min) - transcription CONTINUES past this.

The truncated flag must read False on every run.

Validated against a purpose-built file: spoken audio, then 16 minutes of silence, then a second and unrelated spoken passage. Both passages were transcribed, the break was labelled, and nothing was cut.

Two things worth carrying forward

A silence threshold is a map of the breaks, never a place to stop. Use a level measurement to decide whether there is sound at all. Use gap length only to label where a meeting paused.
Whisper output is not stable run to run. The same 4.89 minute file returned 269, then 313, then 232 words across three passes with identical settings. The words are real, the counts drift. Pin temperature and disable fallback when reproducibility matters.

Measured with ffmpeg volumedetect, ffmpeg silencedetect and faster-whisper base, int8, CPU, on macOS. Method notes only. No transcript content is published here.