🎓 A1Learner
🔍
0 of 48 lessons completed

Julia Benchmarking Method

A benchmark is a careful measurement used to compare how code behaves. It warms code up, measures it, and compares the results across input sizes.

Warm up, then measure one size

build_list is called once as a warm-up call. The measured call for size 10 allocates 144 bytes.

function build_list(n)
    return [i for i in 1:n]
end
build_list(10)
println(@allocated build_list(10))

Measuring across sizes shows growth

The same function is measured at three sizes. The counts grow as the input grows.

function build_list(n)
    return [i for i in 1:n]
end
build_list(10)
println(@allocated build_list(10))
println(@allocated build_list(100))
println(@allocated build_list(1000))

A summing function stays small across sizes

The lists are built before the measurement, and sum_vec is warmed up. The counts stay small at all three sizes.

function sum_vec(a)
    total = 0
    for x in a
        total += x
    end
    return total
end
s10 = collect(1:10)
s100 = collect(1:100)
s1000 = collect(1:1000)
sum_vec(s10)
println(@allocated sum_vec(s10))
println(@allocated sum_vec(s100))
println(@allocated sum_vec(s1000))

Example

function build_list(n)
    return [i for i in 1:n]
end
function sum_vec(a)
    total = 0
    for x in a
        total += x
    end
    return total
end
s10 = collect(1:10)
s100 = collect(1:100)
s1000 = collect(1:1000)
build_list(10)
sum_vec(s10)
println(@allocated build_list(100))
println(@allocated sum_vec(s100))
println(@allocated build_list(1000))
println(@allocated sum_vec(s1000))

Key points

  • Benchmark: a benchmark is a repeated measurement used for comparison.
  • Warm-up step: warm a function up with one call before any measurement.
  • Measure counts: measure with @allocated, and print counts rather than times.
  • Input sizes: measure the same function at several input sizes.
  • Scaling: scaling is how a measurement changes as the input grows.
  • Flat growth: a count that stays small while the input grows shows work that does not grow with the input.

In this example, build_list makes a new list at two sizes, sum_vec adds a ready list at the same sizes, and the counts show building growing while summing stays small.

Note

@allocated counts are stable for fixed code, so a comparison between them is fair.

Try it Yourself

Edit the code, then press Run.

Running is simulated · JuliaHub

Exercise

A benchmark shows 144 bytes at size 10, 928 at size 100, and 8064 at size 1000. What does that scaling show?