🎓 A1Learner
🔍
0 of 47 lessons completed

Julia Parallel Pipeline Project

A parallel pipeline runs a pure function over fixed data on separate processes. The results come back in input order with an integer total.

A pure function maps fixed data to ordered results

Squaring maps each input to its square. The results stay in input order.

square(x) = x * x
results = map(square, [1, 2, 3, 4])
println(results)
println(sum(results))

A second pure function gives a different ordered total

Doubling maps each input to twice its value. The results stay in input order.

double(x) = 2 * x
results = map(double, [4, 7])
println(results)
println(sum(results))

A pure function repeats the same answer

The same input gives the same output on every call. That repeatability is what makes the function safe to run in parallel.

square(x) = x * x
println(square(5))
println(square(5))

Example

using Distributed
addprocs(2)
println(nworkers())
@everywhere square(x) = x * x
results = pmap(square, [1, 2, 3, 4])
println(results)
println(sum(results))

Key points

  • Pure function: a pure function returns the same output for the same input every time.
  • Worker: a worker is a separate process that runs part of the work.
  • Pmap: pmap runs a pure function over fixed data and returns results in input order.
  • Ordered results: ordered results keep the input order even when workers finish at different times.
  • Integer total: an integer total adds the ordered results exactly.

In this example, workers are added first and the worker count is printed. The square function is then shared with every worker and pmap runs it over four fixed inputs. The printed list is in input order and the last line is the integer total.

Note

each code block runs in a fresh process, so workers are added inside the block that uses them.

Try it Yourself

Edit the code, then press Run.

Running is simulated · JuliaHub

Exercise

Pmap runs square over 1, 2, 3, and 4. In what order do the results return?