🎓 A1Learner
🔍
0 of 47 lessons completed

Hot Spots and Rebalancing

A hot spot is one shard that receives far more work than the other shards. A hot spot forms when many requests chase the same popular rows.

One account pulls the reads

This demo sends reads for a famous account and for two quiet accounts. The counts show the reads piling onto a single shard.

requests = ["Famous"] * 90 + ["Ana"] * 5 + ["Ben"] * 5
shard_for = {"Famous": "Shard 1", "Ana": "Shard 2", "Ben": "Shard 3"}
counts = {"Shard 1": 0, "Shard 2": 0, "Shard 3": 0}
for name in requests:
    counts[shard_for[name]] = counts[shard_for[name]] + 1
for shard in ["Shard 1", "Shard 2", "Shard 3"]:
    print(shard + ":", counts[shard], "reads")

The hot shard also holds the most rows

This demo prints the rows held by each shard in the same system. The busiest shard holds far more rows than the others.

shards = [list(range(1, 601)), list(range(601, 801)), list(range(801, 1001))]
number = 1
for shard in shards:
    print("Shard", number, "holds", len(shard), "rows")
    number = number + 1

Splitting the hot shard

This demo splits the range of the hot shard in half. The counts show how many rows stay and how many rows move.

hot = list(range(1, 601))
stay = hot[0:300]
move = hot[300:600]
new_shard = []
for row in move:
    new_shard.append(row)
print("Rows that stay on the hot shard:", len(stay))
print("Rows moved to the new shard:", len(new_shard))
print("First moved row:", new_shard[0])
print("Last moved row:", new_shard[-1])

Example

requests = ["Famous"] * 90 + ["Ana"] * 5 + ["Ben"] * 5
shard_for = {"Famous": "Shard 1", "Ana": "Shard 2", "Ben": "Shard 3"}
counts = {"Shard 1": 0, "Shard 2": 0, "Shard 3": 0}
for name in requests:
    counts[shard_for[name]] = counts[shard_for[name]] + 1
for shard in ["Shard 1", "Shard 2", "Shard 3"]:
    print(shard + ":", counts[shard], "reads")
hot = list(range(1, 601))
stay = hot[0:300]
move = hot[300:600]
new_shard = []
for row in move:
    new_shard.append(row)
print("Rows on the hot shard before the split: 600")
print("Rows moved to the new shard:", len(new_shard))
print("Rows left on the hot shard:", len(stay))
print("Rows on the new shard:", len(new_shard))

Key points

  • Hot spot: one shard carries most of the work while the other shards stay light.
  • Popular rows: a famous account or a best-selling item can draw far more reads than a typical row.
  • Rebalancing: rows move from a busy shard to a lighter shard so the work spreads out.
  • Use a split of the hot shard to move part of its rows onto a new shard.

In this example, one popular account pulls most reads onto one shard, and that shard is then split. Half of its rows move to a new shard, and the moved count shows the copy work that rebalancing must do.

Note

Rebalancing is not free, because the moved rows must be copied to their new home.

Try it Yourself

Edit the code, then press Run.

Running is simulated

Exercise

One shard receives far more reads than the others because it holds a famous account. What is this called?