Running LLama 2 70b a Mac Studio gets 13 tokens per second.

2 x 4090 is more like 38 tokens per second.

But it's harder than that. As the 2 x 4090 is using https://llm.mlc.ai/ which is highly optimized so perhaps the Mac Studio inference could be better when tuned.

Then a lot of production inference engines use continous batching, which means they perform really well when processing multiple prompts at a time.