I've looked into it and the only sane answer right now is still "If it flies, floats, or infers, rent it." You need crazy high memory bandwidth for good inference speed, and that means GPUs which are subject to steep monopoly pricing. That doesn't look to be changing anytime soon.

Second place is said to be the latest Macs with lots of unified memory, but it's a distant second place.

The recently announced hardware from nvidia is either underpowered, overpriced, or both, so there's not much point waiting for it.