They aren’t really an alternative to anything. For one thing they’re now often slower on per-accelerator basis than NVIDIA stuff. They’re cheaper, of course, but because of disparity in performance you’ll need to estimate cost per flop on your own particular workload. They are also more difficult and slower to develop against, and SWE cost is always an issue if you don’t own a money printer like Google. Furthermore, for advanced users who can do their own CUDA kernels or Triton, that too can unlock additional efficiency from GPU. Such capability can’t even be contemplated on the TPU side because you basically get a black box. Then there’s the issue of limited capacity, further exacerbated by the fact that this capacity is provided by a single supplier who is struggling to fulfill its internal needs (which is why you can’t get v5). You can’t just get TPUs elsewhere. You can’t get them under your desk for dev work either.

That said, it wouldn’t be too difficult to port most models to Jax, load in the existing weights, and export the result for serving. Should you bother? IMO, no, unless we’re talking really large scale inference. Your time and money are almost certainly better spent iterating on the models.