🎓 A1Learner
🔍
0 of 47 lessons completed

Caching: Where and Why

A cache is a small, fast store that keeps copies of answers close to the work that needs them. A cache sits between an app server and its database, so repeat requests can skip the database.

A miss first, then hits

The first request for a key misses, finds nothing, and reads the database. Later requests for the same key hit the copy now stored.

db = {"photo-7": "sunset"}
cache = {}
reads = 0

def get(key):
    global reads
    if key in cache:
        print(key, "served from cache")
        return cache[key]
    reads += 1
    print(key, "served from database")
    cache[key] = db[key]
    return db[key]

get("photo-7")
get("photo-7")
get("photo-7")
print("database reads:", reads)

10 requests, 3 keys

10 requests ask for only 3 different keys. Each new key costs one database read, and every repeat is a hit.

db = {"a": 1, "b": 2, "c": 3}
cache = {}
reads = 0
hits = 0
requests = ["a", "b", "a", "c", "b", "a", "c", "a", "b", "a"]

for key in requests:
    if key in cache:
        hits += 1
    else:
        reads += 1
        cache[key] = db[key]

print("requests:", len(requests))
print("database reads:", reads)
print("cache hits:", hits)

A cache inside each server

Each app server here keeps its own cache. The same key misses once per server, then hits on that server.

db = {"title-1": "Welcome"}
caches = {"server 1": {}, "server 2": {}}
reads = 0

for name in ["server 1", "server 2", "server 1"]:
    cache = caches[name]
    if "title-1" in cache:
        print(name, "served from cache")
    else:
        reads += 1
        cache["title-1"] = db["title-1"]
        print(name, "served from database")

print("database reads:", reads)

Example

db = {"hat": 25, "scarf": 15, "gloves": 12, "coat": 80}
cache = {}
reads = 0
hits = 0
requests = ["hat", "coat", "hat", "scarf", "hat", "gloves", "coat", "hat"]

for item in requests:
    if item in cache:
        hits += 1
    else:
        reads += 1
        cache[item] = db[item]

print("requests:", len(requests))
print("database reads:", reads)
print("cache hits:", hits)

Key points

  • Cache hit: a request that finds its answer already waiting in the cache.
  • Cache miss: a request that finds nothing in the cache and must read the database.
  • Place a cache inside an app server to serve data that one server reads again and again.
  • Place a cache as its own store when several app servers should share the same copies.

In this example, 8 requests for shop prices needed only 4 database reads, because the other 4 answers came from the cache. Repeat requests are where a cache earns its place.

Note

a cache holds copies only, so the database stays the home of the real data.

Try it Yourself

Edit the code, then press Run.

Running is simulated

Exercise

A request checks the cache, finds nothing there, and reads the database. What just happened?