With the Claude bug, or so it is known, burning through tokens at record speed, I gave alternative models a try and they're mostly ... interchangeable. I don't know how easy switching and low brand loyalty and fast markets will play out. I hope that local LLMs will become very viable very soon.
Mathematics is hardly an edge case, but SOTA models differ wildly in their ability to write proofs for unsolved problems.
Models also differ wildly in tasks like decompilation for reverse engineering.
Also, so far, the only model I've found which can competently write PTX for SM100 CUDA devices is GPT-5.4pro, but I'm willing to admit that this is more of an edge case than the aforementioned.
AFAICT, the extent to which someone finds models interchangeable is inversely proportional to the novelty of their work.