- One woman can make a baby in nine months - Nine women can make nine babies in nine months - But nine women can't make a baby in one month
Each step of fetal development depends on the previous step. For a given single baby, you can't have one woman working on step #49 while another woman works on step #17.
A lot of computational tasks are similar. Think of the Fibonacci sequence: the computation of each number depends on previous results.
Reason #2: The overhead of communication and coordination. If you split up a task amongst 32 cores, those cores need to communicate with the parent process and perhaps with each other as well. This eats into your transistor budget.
Reason #3: Resource contention. It's nice that you can split up your memory bandwidth-intense task across 128 cores, but if most of those cores are just sitting around and idling because of memory bandwidth contention issues, you haven't gained anything.
Reason #4: It's hard. Programmers struggle to write parallel code. Some tools make it much easier of course.