URL Shortener: Estimate the Scale
Estimation is rough arithmetic that turns daily counts into per-second rates and storage sizes. This lesson estimates the scale, the second step of the method.
From a day to a second
A daily count becomes a per-second rate by dividing by 86,400, the seconds in a day. The printout shows 100,000,000 writes a day becoming 1,157 per second.
writes_per_day = 100000000
seconds_per_day = 86400
print("writes per day:", writes_per_day)
print("seconds per day:", seconds_per_day)
print("writes per second:", round(writes_per_day / seconds_per_day))Reads outnumber writes
Redirects run at 10 per write in this design. The printout shows 1,000,000,000 redirects a day and 11,574 per second.
writes_per_day = 100000000
reads_per_day = writes_per_day * 10
print("reads per write:", reads_per_day // writes_per_day)
print("reads per day:", reads_per_day)
print("reads per second:", round(reads_per_day / 86400))Storage adds up
Each record takes 100 bytes, for the code, the long link, and the date. The printout grows one day of records into a year and into 5 years.
records_per_day = 100000000
bytes_per_record = 100
bytes_per_day = records_per_day * bytes_per_record
print("bytes a day:", bytes_per_day)
print("GB a day:", bytes_per_day // 1000000000)
print("TB a year:", bytes_per_day * 365 / 1000000000000)
print("TB in five years:", bytes_per_day * 365 * 5 / 1000000000000)Example
writes_per_day = 100000000
reads_per_day = writes_per_day * 10
bytes_per_day = writes_per_day * 100
print("writes per second:", round(writes_per_day / 86400))
print("reads per second:", round(reads_per_day / 86400))
print("reads per write:", reads_per_day // writes_per_day)
print("GB of new records a day:", bytes_per_day // 1000000000)
print("TB of new records a year:", bytes_per_day * 365 / 1000000000000)Key points
- Writes: the shortener takes 100,000,000 new links a day, about 1,157 writes per second.
- Reads: redirects run at 10 per write, so the shortener serves 1,000,000,000 redirects a day, about 11,574 per second.
- Record size: one stored record takes 100 bytes, for the code, the long link, and the date.
- Daily storage: new records take 10 GB a day.
- Long-term storage: new records take 3.65 TB a year and 18.25 TB over 5 years.
In this example, the shortener takes 1,157 writes per second and 11,574 redirects per second. Storage grows by 10 GB a day, so the read path and the store both shape the design that follows.
Note
reads dwarf writes in this design, so the redirect path matters most.
Try it Yourself
Edit the code, then press Run.
Running is simulated
Exercise
About how many redirects must the shortener serve per second?