Utilizing multiple CPUs in Julia
When you don’t want other core sitting idle.
A Youtube video about parallelism
Before going multi-core, improve the serial performance first!
High level packages
Recommended for regular users.
- Folds.jl with a unified interfae for a variety of executers (multithreading, multiprocessing, CUDA)
- FLoops.jl with
- ThreadsX.jl for parallelized Base functions.
Low level constructs
Task-based asynchronous programming
Multithreading within in a CPU core, a.k.a “Green threading”.
- Caveats in multithreading
- Threads share memory causing data racing if data access is not properly handled.
- Suitable for SMP (Symmetric multiprocessing)
To increase the threads avaiable to Julia, start Julia with
-t / --threads argument. e.g.
Or through an environment variable,
JULIA_NUM_THREADS=n. For example in
You could check how many thread are available in the Julia session
Enable multithreading by
Threads.@threads for loop
- Suited for clusters
- Independent memory pool by default, except for shared memories.
- Unlike Python, where multiprocessing is done by forking and worker processes inherit data from the main process.
- As a result,
@everywhereis needed for code running in parallel to copy the data to the worker processes.
A simple example to start julia with multiple processes:
To load files for all processes without