🎓 A1Learner
🔍
0 of 47 lessons completed

Choosing a Shard Key

A shard key is the value a system uses to decide which shard holds each row. A good shard key spreads rows evenly across the shards.

Placing users by id number

This demo places each user by id number spread. The counts show the shards filling evenly.

users = list(range(1, 91))
counts = [0, 0, 0]
for user in users:
    counts[user % 3] = counts[user % 3] + 1
number = 1
for count in counts:
    print("Shard", number, ":", count, "users")
    number = number + 1

Placing users by signup month

This demo places the same users by signup month. The counts show one month piling most users onto a single shard.

months = ["June"] * 70 + ["May"] * 10 + ["July"] * 10
shard_for_month = {"May": 1, "June": 2, "July": 3}
counts = [0, 0, 0]
for month in months:
    counts[shard_for_month[month] - 1] = counts[shard_for_month[month] - 1] + 1
number = 1
for count in counts:
    print("Shard", number, ":", count, "users")
    number = number + 1

Both keys on the same users

This demo runs both keys over the same group of users. The two printed spreads show the key deciding the balance.

users = list(range(1, 91))
months = ["June"] * 70 + ["May"] * 10 + ["July"] * 10
shard_for_month = {"May": 0, "June": 1, "July": 2}
by_id = [0, 0, 0]
by_month = [0, 0, 0]
for user in users:
    by_id[user % 3] = by_id[user % 3] + 1
for month in months:
    by_month[shard_for_month[month]] = by_month[shard_for_month[month]] + 1
print("By id number:", by_id)
print("By signup month:", by_month)

Example

users = list(range(1, 301))
months = ["June"] * 240 + ["May"] * 30 + ["July"] * 30
shard_for_month = {"May": 0, "June": 1, "July": 2}
by_id = [0, 0, 0]
by_month = [0, 0, 0]
for user in users:
    by_id[user % 3] = by_id[user % 3] + 1
for month in months:
    by_month[shard_for_month[month]] = by_month[shard_for_month[month]] + 1
print("Users placed by id number:", by_id)
print("Users placed by signup month:", by_month)
print("Fullest shard by id number:", max(by_id))
print("Fullest shard by signup month:", max(by_month))

Key points

  • Shard key: the field whose value decides the shard for a row.
  • Even spread: a good key places about the same number of rows on every shard.
  • Skew: a poor key places most rows on one shard and leaves the other shards light.
  • Use a value that varies widely from row to row to decide the shard.
  • Signup month: a month groups many users who joined together, so a month can pile users onto one shard.

In this example, a larger group of users is placed by id number and then by signup month. The id number spread stays even, and the month spread leaves one shard holding most of the users.

Note

The data stays the same when the key changes, and only the placement changes.

Try it Yourself

Edit the code, then press Run.

Running is simulated

Exercise

Which shard key spreads users most evenly across the shards?