Hacker News Reader

Comment by xfalcox | Hacker News Reader

You'd be surprised how often people in enterprise can be left waiting months to get an API key approved for an LLM provider.

View on HN

Are you saying that it's faster for them to get the hardware to run the weights themselves? Otherwise I'm not sure what the relevancy is.

Yes some have existing infra

I'm having a somewhat hard time believing a corporation where getting a API key for a LLM service is very difficult, somehow has the (GPU) infrastructure already running for doing the same thing themselves, unless they happen to be a ML corporation, but I don't think we're talking about those in this context.

Nah this is definitely a real scenario. Getting access to public models requires a lot of security review, but proving through Bedrock is much more simple. I may be spoiled in having worked for companies that have ML departments and developer XP departments though.

Not sure Bedrock counts as self-hosting though, isn't it a managed service Amazon provides?

> I may be spoiled in having worked for companies that have ML

Sounds likely, yeah, how many companies have ML departments today? DS departments seem common, but ML i'm not too sure about

A lot of companies think they do

No, this is very real. One reason why this can happen: a company has elaborate processes for protecting their internal data from leaking, but otherwise lets engineers do what they want with resources allocated to them.

Unless they are already in the possession of such hardware (like an M3 mac, for example).

Reply on news.ycombinator.com