🎓 A1Learner
🔍
0 of 45 lessons completed

Benchmarking Notes

Benchmarking is measuring how fast a program runs. Honest benchmarking measures the real work, not the setup around it.

Warm up before you measure

The first run can be slower than later runs. This program runs a small sum once as a warm-up, then runs it again as the measured work.

@main def main(): Unit =
  val numbers = List(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
  val warm = numbers.sum
  val measured = numbers.sum
  println(warm)
  println(measured)

Repeat the work many times

One run can be noisy. This program runs the same sum five times and prints the repetition count and the final result.

@main def main(): Unit =
  val numbers = List(1, 2, 3, 4, 5)
  var result = 0
  var count = 0
  while count < 5 do
    result = numbers.sum
    count = count + 1
  println(count)
  println(result)

Measure the work, not the setup

Setup builds the list before the counted work starts. This program builds a list first, then counts only the sum step in its repetition count.

@main def main(): Unit =
  val numbers = (1 to 100).toList
  var result = 0
  var runs = 0
  while runs < 3 do
    result = numbers.sum
    runs = runs + 1
  println(numbers.size)
  println(runs)
  println(result)

Example

@main def main(): Unit =
  val numbers = (1 to 20).toList
  val warm = numbers.sum
  var result = 0
  var runs = 0
  while runs < 4 do
    result = numbers.sum
    runs = runs + 1
  println(warm)
  println(runs)
  println(result)

Key points

  • Warm-up: A short practice run before measuring lets the program settle.
  • Repetition: Running the same work many times smooths out single-run noise.
  • Setup: Work done before timing starts must stay outside the measured part.
  • Work: The measured part is only the code whose speed you want to know.
  • Result: A benchmark result is useful only when the same run can be repeated.

In this example, the list is built first, then a warm-up sum runs once, then the sum runs four more times, and the program prints the warm-up result, the repetition count, and the final result. Real wall time would be measured around the repeated part, outside the printed evidence.

Note

Scala code on the JVM warms up, so a first run is often slower than later runs.

Try it Yourself

Edit the code, then press Run.

Running is simulated · Scastie

Exercise

Which plan measures honestly?