Latency and Throughput
Latency is the time one request takes, and throughput is the count of requests finished in a set time. One fast request says nothing about how many requests finish overall.
Fastest and slowest waits
Each request in the list starts at the same moment, and each time runs from its own start to its own finish.
times_ms = [80, 120, 450, 900, 1500, 2200]
print("Fastest request:", min(times_ms), "ms")
print("Slowest request:", max(times_ms), "ms")Finished inside one second
One second is 1,000 milliseconds, and a request counts when its time is no more than that.
times_ms = [80, 120, 450, 900, 1500, 2200]
finished = 0
for time_ms in times_ms:
if time_ms <= 1000:
finished = finished + 1
print("Finished in one second:", finished)
print("Throughput:", finished, "requests per second")Example
times_ms = [80, 120, 450, 900, 1500, 2200]
finished = 0
for time_ms in times_ms:
if time_ms <= 1000:
finished = finished + 1
average_ms = sum(times_ms) / len(times_ms)
print("Requests:", len(times_ms))
print("Fastest latency:", min(times_ms), "ms")
print("Slowest latency:", max(times_ms), "ms")
print("Average latency:", average_ms, "ms")
print("Throughput in first second:", finished, "per second")Key points
- Fastest latency: the smallest latency in a set of requests.
- Slowest latency: the largest latency in a set of requests.
- Average latency: add the latencies in a set, then divide by how many requests there are.
- Fixed window: a throughput count names its time window, such as one second.
- Worked result: 4 requests finish inside the first second in the demo data.
In this example, the latencies come from single requests in the list. The throughput instead counts how many of those same requests finish inside one second.
Note
a fast average can still hide one very slow request in the same set.
Try it Yourself
Edit the code, then press Run.
Running is simulated
Exercise
One request took 900 milliseconds from start to finish. What does that single number measure?