Pre-Emptive scaling ala erlang can help with scenario one somewhat, if the jobs aren’t locked on some resource. For example, on my erlang system 20 would all run just each slightly slower as they get a smaller amount of scheduler reductions each.

It’s a hard/interesting problem, and harder still once you’re running across a lot of machines — but if they all get slower under the load it turns out that it’s easier to scale / work out a good balance of idle capacity to guarantee x time sla under x requests.

Fast ramp up for additional capacity is important too, but less so if you know to start the process once median execution time drops to some % of your worst target