Having both written physics engines and worked with GPUs extensively (including GPU physics simulation), I would be really hesitant to go GPU-first unless there was a very specific use case in mind.

There's no doubt that GPUs have the flops/bandwidth advantage, and even all the overheads of adapting a physics engine to a GPU won't completely eliminate those advantages. If a company needed an absolute-maximum-performance backend simulation on computers they built to spec, it could definitely make sense.

But when you move into the realm of game client simulation, it's harder to justify the use of GPUs. Very, very few gamers have multiple GPUs, and the one GPU they do have is often already quite busy.

Ignoring all the nightmarish issues around driver bugs and vendor compatibility, it comes down a question of comparative advantage. GPUs are extremely good at graphics. They can also do a pretty darn good job at physics. Or sound path tracing. Or machine learning inference. Or any number of other high throughput workloads. There's a limited budget, and there's no reason to leave a beefy CPU idle. What task allocation gives you the most bang for the buck? Would your average gamer be okay with worse graphics, just because the GPU is needed to perform physics simulation in order to play the game? The answer will sometimes be yes, but it's something to be careful about.

Plus, especially with the latest surge in CPU performance and competition courtesy of AMD, CPUs can do a pretty darn good job of running physics simulations per dollar.