>
This book is notable for its coverage of MPI and OpenMP in both C, Fortran, C++, and (for MPI) Python."The Art of HPC", volume 2 > "Parallel Programming for Science Engineering" https://theartofhpc.com/pcse/index.html
FWIW, MPI is only one way to Python for HPC.
ipyparallel will run MPI jobs over tunnels you create yourself IIRC.
A chapter on dask-scheduler, CuDF, CuGraph (NetworkX), DaskML, and CuPy, and dask-labextension would be more current.
Dask doesn't handle data storage for you, so it's your responsibility to make sure that the data store(s) before each barrier are not the performance bottleneck.
Dask docs > High Performance Computers:
https://docs.dask.org/en/stable/deploying-hpc.html
Sources of random may be the bottleneck. You don't know until you profile the job across the cluster.
Re: eBPF-based tracing tools: https://news.ycombinator.com/item?id=31688180
And then something about GitOps (and ChatOps), code review and revision, and project resource quotas