Whisper Transcribed 60 Seconds of Silence as "Thank You."

I'm building a voice journal. You talk for sixty seconds, it writes the entry in your words. Which means the one thing the pipeline must never do is put words in your mouth.

This week I fed it sixty seconds of pure digital silence — not a quiet room, actual zeroed audio — and Whisper returned, with total confidence: "Thank you. Thank you."

Eight seconds of silence: "Thank you." Faint noise: a single period. If you've ever wondered why auto-subtitles end with a phantom "thanks for watching," this is why. Whisper was trained on oceans of subtitled video, and when it hears nothing, it reaches for how subtitled videos end. It answers silence with boilerplate — fluently, confidently, every time.

My pipeline saved those as real journal entries. A journal that invents an entry you never spoke is worse than one that loses an entry. So I went looking for a reliable way to tell silence from speech, and everything obvious failed in an instructive way.

The confidence field that carries no signal

The obvious fix: Whisper returns a no_speech_prob field. Gate on it, done.

Measured on Groq's whisper-large-v3-turbo: sixty seconds of pure digital silence scores no_speech_prob = 0.000. Real speech also scores 0.000. The field with "no speech probability" in its name carries no signal at all for exactly the case its name describes.

That's the first rule I'm keeping from this: never trust a confidence field's name. Measure what it actually returns on the failure case you care about, before you design around it.

The blocklist that lost to a repeat

Second attempt: a blocklist of known hallucinations — "thank you", "thanks for watching", ".". Whole-transcript match, so a real entry that merely contains "thank you" is safe.

It held for a day. Then sixty seconds of silence came back as "Thank you. Thank you." — the same hallucination, twice. A whole-transcript match can't catch a repeated one. And a per-sentence check turned out to be needed anyway, because the hallucination vocabulary is bigger than any list you write on day one.

The trap I set for myself

Third failure, and my favorite: to make Whisper spell my product names right, I pass a vocabulary prompt — example sentences containing "SayTrail" so the decoder has a prior. Whisper conditions on that prompt as if it were the preceding transcript.

So what does it do with four seconds of faint noise? It echoes the prompt back. My anti-typo vocabulary came out as the transcription: a sentence I wrote, filed as words the user spoke. The richer you make the priming prompt, the better the echo. Your own scaffolding becomes the hallucination — and by definition it's never on your blocklist.

The numbers that finally separated them

I stopped guessing and measured avg_logprob across the matrix, vocabulary prompt attached, on whisper-large-v3-turbo:

• -60s pure silence: -0.577
• -10s faint noise: -0.620
• -4s noise echoing the prompt: -0.416
• -Worst real speech I could produce (Hinglish, fast, accented): -0.358
• -Quiet 3-second real speech: -0.243
• -Normal speech: -0.089

Look at the gap between the worst real speech and the best-scoring hallucination: 0.06. No single confidence threshold separates them. If confidence alone is your gate, you will either fabricate entries or eat real ones.

The discriminator that actually works is word count. Hallucinations are short; real entries are not. The gate that shipped requires both signals: low confidence AND under eight words. Either alone misfires.

Then the review pass broke even that. A real, exhausted "Okay." scores -0.413. A real "Grateful." scores -0.475 — below the floor I'd calibrated on eighteen-word clips. A threshold justified by one regime's data was firing in a different regime. The fix was two floors: a stricter one for prompt-echo, a looser one for silence, set in the measured gap between a real terse night (about -0.41) and actual silence (about -0.58).

The decision that matters more than the gate

One product decision outranks all the detection work: when the pipeline decides "nothing was heard," it keeps the recording on the device instead of discarding it. Because the gate will eventually false-positive on someone's real, quiet, three-word night — and the order of harms is fixed: fabricating an entry is worse than losing one, and losing one is worse than making someone tap Retry.

If you're persisting Whisper output anywhere — journals, meeting notes, medical dictation — assume it will answer silence with fluent boilerplate, assume your own prompt will come back as an echo, and assume the confidence fields lie. Measure your own failure cases. The numbers above took an afternoon to collect and they changed every design decision that followed.

— Dhruv