NL-2610.006Result[nl.SA]
Using Whisper to check text-to-speech takes
The Nook team
Submitted 5 October 2026 · 2 min read
For a 40-second film introducing Nook, we generated the voiceover locally with Supertonic, an open text-to-speech model, running on audio.cpp. Its output differs from run to run, and some takes stumble. Rather than listen to every take, we transcribed each one with Whisper and kept the take whose transcript matched the script exactly. The same check showed which words the voice gets wrong: "AI" and "GPU" were read correctly, "A.I." was not, and "Nook" in the middle of a sentence often came out as "nuke". The transcripts' word timings then helped time the film's visuals to the voice.
- Comments
- Supertonic 3, voice F1, on audio.cpp; Whisper small on whisper.cpp's CUDA build. The number of takes was not recorded.
- Subjects
- Speech and audio (nl.SA); Methods (nl.MT)
- Cite as
NL-2610.006· usenook.ai/labs/whisper-checks-speech.html
Full text
A speech-to-text model makes a useful first judge of a text-to-speech take: if Whisper hears exactly the script, the take is very likely clean. It also finds the words a voice cannot say.
The problem
We made Nook's 40-second introduction film with a voice generated on our own machine, using Supertonic 3 (voice F1) through audio.cpp, the same engine Nook uses to speak. Text-to-speech models sample as they go, so the same line comes out a little differently each time, and now and then a take slurs a word, drops one, or says something else. Listening to every take of every line is slow, and ears tire.
The check: transcribe it back
- Generate several takes of each line.
- Transcribe every take with Whisper (the small model, through whisper.cpp on the graphics card).
- Keep the take whose transcript matches the script exactly.
Because Whisper also gives a time for every word, the same transcripts told us when each word is spoken, and the film's visuals were timed to the voice with them.
What tripped the voice
| Written | Spoken |
|---|---|
| AI | Read correctly |
| GPU | Read correctly |
| A.I. | Not read correctly |
| Nook, in the middle of a sentence | Often “nuke” |
So write “AI”, never “A.I.”, and give lines with “Nook” in the middle more takes, or put the name at the start.
An engine detail
audio.cpp can take many requests in one run (--request-sequence), each with its own options. Setting speaking_rate that way had no effect for us, while the same setting as a flag on a single-request run did work. If a setting seems to be ignored in a sequence, try it on one request.