Audio Course

Work out how much audio you are really feeding a model.

1. Take a 30-second clip. At 16 kHz mono that is 480,000 raw samples — far too many to attend over directly.
2. That is why models use spectrograms: the waveform becomes a time-frequency image, typically ~50 frames per second.
3. Recompute: 30 seconds ≈ 1,500 frames. Now it is a sequence length a transformer can handle.
4. Check your source sample rate. Feeding 44.1 kHz audio to a model expecting 16 kHz silently degrades accuracy.

What you should see: The resampling step is the one most often skipped, and it fails quietly — you get worse transcripts, not an error.

Measure word error rate on your own audio, not on a benchmark.

1. Take three clips that represent your hardest real cases: background noise, an accent, and domain jargon.
2. Transcribe each, then hand-write the correct transcript.
3. Count substitutions, deletions and insertions, and divide by the number of reference words. That is WER.
4. Look specifically at the jargon: names and technical terms fail far more often than the headline number suggests.

What you should see: Your WER will be worse than the published figure, because benchmarks are clean read speech. The jargon errors are usually the ones that matter to users.

Test the cases where synthesis actually breaks.

1. Synthesise an ordinary sentence. It will almost certainly sound fine — this tells you little.
2. Now try the hard inputs: an abbreviation, a number like 1,024, a date, a URL, and a name from another language.
3. Listen for whether "1,024" is read as a number or as digits, and whether the abbreviation is expanded or spelled.
4. Add a question mark and an exclamation mark to the same sentence and check the intonation changes.

What you should see: Naturalness fails on normalisation — numbers, dates and abbreviations — long before it fails on phonemes. That is where to spend your evaluation effort.

Decide between batch and streaming before you build.

1. Time a full-file transcription of a 60-second clip end to end.
2. Compute the real-time factor: processing time ÷ audio duration. Below 1.0 means faster than real time.
3. If you need live captions, batch will not do — the user waits for the whole file. You need chunked streaming with overlap.
4. If you are processing recordings, batch is simpler and usually cheaper. Choose deliberately.

What you should see: A real-time factor under 1.0 is necessary but not sufficient for live use: latency is felt per chunk, and chunk overlap costs extra compute.