It's a good point, and the way my library is designed now it would be a big resource hog. Not sure the best way to handle that given that it's supposed to be a super easy to use library for Python. Perhaps I could add an optional parameter "no_resource_hogging" or something, and if enabled that can ratchet down the amount of parallelism, or reduce the CPU priority of the threads to a very low level so it won't use up all the system resources. The problem with the latter approach is that it starts being system specific. For example you could do something like this:
fn create_low_priority_pool() -> rayon::ThreadPool {
ThreadPoolBuilder::new()
.start_handler(|_| {
#[cfg(target_os = "linux")]
{
use libc::{sched_param, sched_setscheduler, SCHED_IDLE};
unsafe {
let mut param: sched_param = std::mem::zeroed();
param.sched_priority = 0;
sched_setscheduler(0, SCHED_IDLE, ¶m);
}
}
#[cfg(target_os = "windows")]
{
use winapi::um::processthreadsapi::SetThreadPriority;
use winapi::um::winbase::THREAD_PRIORITY_LOWEST;
unsafe {
SetThreadPriority(std::ptr::null_mut(), THREAD_PRIORITY_LOWEST);
}
}
})
.build()
.unwrap()
}
But I don't know how that would work for MacOS. Anyway, it's something to think about.