> If I have a choice of making single job faster vs just running 64 jobs in parallel on CPU I'd pick the second every time I can because code will be simpler and less buggy every single time.

I maybe misunderstanding what you're trying to say, don't you mean you'd pick the first one every time? Make the single job faster instead of introducing concurrency with 64 parallel jobs?