Ask HN: My process only gets scheduled on M1 efficiency cores

Not sure where else to post

I work on some CPU intensive software written in C++ and multithreaded with with OpenMP. This software has not been scaling how I would expect on my M1 Max MacBook. Notably, when I use 10 threads I see in activity monitor that the process has the expected number of threads but only seems to be getting scheduled on the the two efficiency cores for long runs. This is not ideal.

I’m not sure where to begin diagnosing this issue. Some relevant information:

* I’ve been using the llvm-13 toolchain packaged by home-brew since the toolchain included with macOS does not support openMP

* Simple OpenMP for loops behave how I would expect and seem to get scheduled to all cores. The issue seems to be for long-running processes.

* For the project in question, the runtime bottle neck of the code is the OpenMP accelerated solvers from AMGCL, and this is where I get hit with the only-getting-scheduled-on-efficiency cores issue.

    \* https://github.com/ddemidov/amgcl

    \* There doesn’t seem to be any unusual OpenMP primitives in AMGCL

* Using OMP_NUM_THREADS to use less than 10 threads (like 2 - 6), the process sometimes gets scheduled on performance cores, but this does not seem to be a guarantee.

What are some tools / leads for diagnosing why my the simulation process tends to get scheduled onto efficiency cores? What are some options for changing how my process get scheduled?