🎓 A1Learner
🔍
0 of 47 lessons completed

Julia Parallel Design

Parallel design is choosing a pattern by the shape of the work. In other words, designing a parallel program starts with the shape of the work, not with the tool.

Independent elements suit a thread loop

Numbers from 1 to 20 are independent, so a thread loop with partial totals adds them exactly.

function design_threads()
    parts = zeros(Int, Threads.nthreads())
    Threads.@threads for i in 1:20
        parts[Threads.threadid()] += i
    end
    return sum(parts)
end
println(design_threads())

Uneven work suits tasks and a channel

Four jobs travel through a channel, and two workers share the taking.

function design_channel()
    jobs = Channel{Int}(10)
    for v in [10, 20, 30, 40]
        put!(jobs, v)
    end
    close(jobs)
    partials = [0, 0]
    @sync begin
        @async for job in jobs
            partials[1] += job
        end
        @async for job in jobs
            partials[2] += job
        end
    end
    return sum(partials)
end
println(design_channel())

One shared value suits an atomic integer

Numbers from 1 to 10 update one atomic integer, and the final total is exact.

function design_atomic()
    counter = Threads.Atomic{Int}(0)
    Threads.@threads for i in 1:10
        Threads.atomic_add!(counter, i)
    end
    return counter[]
end
println(design_atomic())

Example

function thread_shape()
    parts = zeros(Int, Threads.nthreads())
    Threads.@threads for i in 1:10
        parts[Threads.threadid()] += i
    end
    return sum(parts)
end
function shared_shape()
    counter = Threads.Atomic{Int}(0)
    Threads.@threads for i in 1:10
        Threads.atomic_add!(counter, i)
    end
    return counter[]
end
println(thread_shape())
println(shared_shape())

Key points

  • Data shape: use @threads or @distributed for independent elements that need the same step.
  • Task shape: use @async and channels for uneven work that arrives at different times.
  • Separate process shape: use distributed workers when work needs separate processes.
  • Shared state shape: use a lock or an atomic integer when threads update one shared value.
  • Same result rule: choose the pattern that keeps the result the same on every run.

In this example, a thread loop with partial totals adds the numbers from 1 to 10, then an atomic integer adds the same numbers, and the example prints both totals.

Note

the fastest pattern is not always the right one, because a pattern must fit the shape first.

Try it Yourself

Edit the code, then press Run.

Running is simulated · JuliaHub

Exercise

Which shape suits a thread loop with partial totals?