Design a Video Conferencing System

The ask

A meeting joins a set of participants for live voice and video. A host creates a meeting and shares its link.

Participants join, speak, share a screen and leave. Chat and recording sit beside the live media.

Home connections vary widely. The meeting must stay usable on a weak connection.

Delay hurts conversation. Media must take a short path from sender to receivers.

Estimate the scale

10,000,000 meetings happen a day.

10,000,000 / 86,400 = 116 meetings per second.

The average meeting has 10 participants and lasts 40 minutes.

10,000,000 x 10 x 40 = 4,000,000,000 participant-minutes a day.

One video stream uses 1.5 megabits per second. Bandwidth, not storage, is the scarce resource.

API and data

The create operation takes the host, the start time and the invited participants. The response returns the meeting link.

The join operation takes the meeting link and the participant's credentials. The response returns the address of the assigned media server.

A meeting record holds the host, the scheduled times, the participant list and the recording state. Media itself is never stored in the record.

Chat messages and attendance are stored as meeting data. Recordings go to blob storage when the host enables them.

High-level design

Signalling is the exchange of small control messages that set up, change and end a call. A signalling service handles joins, leaves and mute state.

Media servers carry the actual voice and video. Each meeting is placed on one media server in a nearby data center.

A placement service picks that server by participant locations and current load. Most participants sit near the chosen region in the common case.

Deep dive: media forwarding

Media forwarding is the pattern where one server receives each stream and forwards a copy to every other participant. Senders upload once.

Without a server, each sender would upload a separate copy to every other participant. Home upload links would collapse first.

A media server receiving 10 streams forwards each to the other 9 participants.

10 x 9 = 90 streams out of the server for that one meeting.

90 x 1.5 = 135 megabits per second out for that one meeting.

The server forwards packets without decoding them. Forwarding stays cheap enough to run at line rate.

Uneven connections

Each sender also sends a low-rate version of its stream. A receiver on a weak link is forwarded the low version.

Voice is protected first. Under strain the server drops video quality before it touches sound.

Receivers report loss and delay. The server adjusts which version each receiver gets, continuously.

Large meetings and failure

In a large meeting most participants are silent. The server forwards the active speakers and small still images for the rest.

A media server failure ends its meetings' media at once. Clients rejoin automatically onto a fresh server.

Signalling state survives the move, so the roster and chat stay intact. The gap is a brief freeze, not a lost meeting.

Evaluation and trade-offs

A video conferencing system is a signalling service plus a fleet of forwarding servers. Placement and forwarding carry the design.

Forwarding costs server bandwidth on every stream. Direct pairs would cost home upload links far more.

One server per meeting keeps forwarding simple. Very large meetings spread across servers and pay a coordination cost.

A nearby server cuts delay and may sit far from some participants. Placement balances the whole group, not one member.

Conferencing scale