Hacker News Reader

Comment by capableweb | Hacker News Reader

Unsurprisingly, exactly the same pricing as OpenAI. So what ends up the benefits? I'm currently using GPT-4 for hacking around, and love it if I could get faster response times, would make the use case I'm hacking on a lot more attractive. If I use Microsofts API, would it be any faster compared to OpenAI?

View on HN

Your app could be slightly but meaningfully faster (depending on your use case) if the other servers/services your app relies on are close by (also on Azure, in the same region region or datacenter). Otherwise probably not much more speed difference aside. There are other benefits such as managing everything in the same cloud, single billing, and maybe even some security benefits of having everything in the same VPC. This is assuming Azure will allow you to chose a region for the GPT4 service.

When dealing with GPT/LLMs, latency of requests is a minimal part of the time spent, so I don't think geographic location will matter much for that.

Yes, while the GPT API latency will be minimal, whether it's noticeable by users would still depend on the application, use case, and even the location of the client(s). In some cases, the small latencies may still be noticeable for example in gaming, chat apps, or maybe some large enterprise deployment that's tied to specific regions. I agree that for hacking it won't make a significant difference and there's no reason to get on Azure just for GPT - too much overhead and Azure isn't the easiest to figure out to begin with.

Because it's one less thing to manage outside of Azure. Now you can use IaC such as ARM, Terraform, Bicep, to manage the deployment and configuration.

Maybe you've got some Azure credits and can only spend them on OpenAI when packaged as an Azure service?

Too bad ours just expired :( (anyone from microsoft reads this... hmu?)

Almost all European companies will be inclined to use Azure in EU regions to get data privacy guarantees than use OpenAI's API directly.

Reply on news.ycombinator.com