Design Text-to-Speech

The ask

The product reads written text aloud in a chosen voice.

Text-to-speech is that turning of writing into speech.

Callers send text, a voice choice and a language. The answer is an audio file, or a stream of audio.

Typical uses are reading articles aloud, giving spoken replies, and making audio versions of books.

The voice must sound natural. Flat, robotic reading fails the product even when every word is right.

Estimate the scale

The service expects 10,000,000 speech requests a day.

10,000,000 requests ÷ 86,400 seconds = 116 requests per second.

The average clip is 30 seconds of audio.

Generation runs 3 times faster than real time. One clip takes 30 seconds of audio ÷ 3 = 10 seconds of compute.

116 requests per second × 10 seconds = 1,160 clips being generated at any moment.

10,000,000 clips × 30 seconds = 300,000,000 seconds of new audio a day.

Data and features

Training data pairs recorded speech with the exact text that was read.

Many speakers are needed. Accents, ages and speaking speeds all change how the same sentence sounds.

Recording conditions matter. Clean studio recordings teach cleaner sound than noisy rooms.

A voice belongs to a person. Recordings are used only with the speaker's clear permission.

The model learns how words map to sounds. It also learns where a natural voice rises, falls and pauses.

Pronunciation guides are added for names and rare words. The model would otherwise guess, and often guess wrong.

High-level design

A caller sends text and a voice choice to an app server.

The server checks the text length and rejects requests past the stated limit.

A generation job enters a queue, and GPU workers run the model.

Finished audio goes into blob storage, and the caller gets a link.

Repeated text in the same voice, like a fixed greeting, is cached. The cached audio replays without new compute.

Training the model

The model reads the text and plans the sound of the speech, a tiny slice at a time.

It learns from the paired recordings which sound follows each piece of text. A final stage turns the plan into the finished audio.

Training runs on GPUs in batches, over many passes through the data.

A new voice is trained from that speaker's consented recordings. Fine-tuning adapts a trained model to the new voice.

Every voice version is checked for misread words before it serves traffic.

Serving predictions

Serving follows the shared generation shape. A request becomes a job, a GPU makes the audio, and storage delivers it.

10,000,000 clips × 10 seconds = 100,000,000 seconds of compute a day.

Because generation outruns playback, a listener can start hearing a clip before the whole file exists.

The same text in the same voice returns the cached audio. News headlines and greetings repeat often, so the cache saves real compute.

A failed job is retried once. After that it is reported as failed, with the text kept for a later attempt.

Deep dive: starting the sound fast

Listeners hate silence at the start. Interactive uses need the first sound to arrive fast, well under a second.

Long text is split at sentence boundaries. The first sentence is generated and played while later sentences are still being made.

Splitting into tiny pieces causes gaps and odd pauses. Splitting into large pieces delays the start.

Each piece keeps the same voice settings. Tone and speed stay steady across the joins.

If generation falls behind playback, the player pauses briefly. Broken sound is never played.

Keeping voices honest

A voice is part of a person's identity. Cloning a voice needs recorded permission from that person.

Requests to imitate a real person without permission are refused.

Well-known voices are a common abuse target. Extra checks apply before such a voice is used.

Generated speech in sensitive settings should be marked as generated. A call that asks for money is one such setting.

A caller can report misuse. Reports are reviewed by people, and abusive accounts are removed.

Evaluation and trade-offs

Listeners rate samples for naturalness and clarity. A clip that is hard to understand fails even if it sounds smooth.

Listening tests compare versions on the same fixed text set. Names, numbers and dates are included, because they are easy to misread.

The misread rate is tracked per language and per voice. A weak language gets more training data before wider release.

Bigger models sound warmer and cost more compute. Faster generation shortens waits and can flatten expression. More voices widen choice and multiply testing.

The scale in numbers