Design Text-to-Image Generation

The ask

The product takes a written description, called a prompt, and returns a newly generated picture.

Text-to-image is that turning of words into a picture.

The first version makes one square picture per prompt. Editing an existing photo stays out of scope.

Answers come back in seconds, not instantly. The interface shows progress while a picture is made.

Estimate the scale

The service expects 100,000,000 image requests a day.

100,000,000 requests ÷ 86,400 seconds = 1,157 requests per second.

One image takes about 10 seconds of GPU time.

1,157 requests per second × 10 seconds = 11,570 images being generated at any moment.

The GPU fleet sets the real capacity. App servers are cheap beside 11,570 pictures in flight.

Data and features

Training data pairs pictures with written descriptions of what they show.

The model learns which visual patterns go with which words.

Data quality decides output quality. Blurry, duplicated or wrongly described pictures teach the wrong links.

Pictures need a clear right to be used. Licensed and public sources are safer than scraped sets with unknown rights.

People check samples of the data by hand. Harmful pictures and wrong descriptions are removed before training.

At run time the prompt is turned into an embedding. The embedding is a list of numbers that captures the meaning of the words.

High-level design

The user sends a prompt to an app server.

A first check rejects prompts that ask for banned content.

The server writes a generation job into a queue and returns a job number at once.

GPU workers take jobs from the queue and run the model.

Finished pictures go into blob storage. The app reads the picture from storage, or from a CDN copy near the user, once the job finishes.

Training the model

The model family used here is diffusion.

Diffusion generates a picture by starting from pure noise. It removes a little noise at each step, guided by the prompt, until a clear picture remains.

Training shows the model millions of picture and description pairs. It learns which noise to remove for a given description.

Training runs in batches on large GPU clusters. It takes far longer than generating one picture.

A new model version is tested offline before it serves traffic. The old version stays live until the new one proves better.

Fresh data and user ratings feed later training runs.

Serving predictions

Take one stored image as 2 MB, counting the picture and its preview copies.

100,000,000 images × 2 MB = 200,000,000 MB a day. That is 200 TB of new images a day, so failed jobs and old previews need a deletion policy.

Workers pull jobs whenever a GPU frees up. No GPU sits idle while the queue holds work.

The user watches a progress state, not a frozen screen. States run from queued, to generating, to ready, to failed.

A finished job returns a link to the stored picture. The picture is served from a CDN, so GPUs never handle repeat downloads.

An exact repeat of a prompt with the same settings can reuse a stored picture. No new GPU time is spent.

Deep dive: the wait for a GPU

The queue is the shock absorber for demand spikes.

100,000,000 images × 10 seconds = 1,000,000,000 GPU-seconds of generation a day. Cost grows in direct proportion to traffic.

When arrivals pass fleet capacity, the queue grows. Waits stretch from seconds to minutes.

Paying users can take a priority lane. Free traffic waits in a slower lane with a stated limit.

Cancelling a queued job is cheap. Cancelling a running job wastes the GPU seconds already spent, so the interface asks once before stopping it.

Cutting the number of noise-removal steps shortens each job. Fewer steps mean faster pictures and rougher detail.

Checking prompts and images

Checks run twice. The prompt is checked before generation, and the finished picture is checked before delivery.

A second model, trained for content moderation, flags banned results.

Borderline results go to human review.

A user can report a picture that slipped through. Reports become labelled examples for the checking model.

Prompts asking for a real person's likeness in a harmful scene are refused.

Evaluation and trade-offs

Human raters score whether a picture matches its prompt. They also score sharpness, faces and general quality.

Offline tests use a fixed prompt set for every model version. A version ships only if it beats the live one on that set.

Online experiments compare versions on real prompts. Ratings and repeat use show whether people like the change.

More noise-removal steps buy quality with time. More GPUs buy shorter waits with money. Stricter checks buy safety with more false refusals.

The scale in numbers