Hacker News Reader

Comment by mushufasa | Hacker News Reader

I would love for things like this to be accessible via hubs like open router or AWS bedrock. It's hard to justify adding new model vendors directly with all the heightened concerns about privacy and security, but if bold new capabilities are added to a centralized already-vendor like AWS, technical people can adopt them without going through a whole compliance/purchasing/vendor review process. And an extra middleman tax is well worth it when the cost savings of the model itself can be one-two orders of magnitude.

View on HN

Isn't openrouter the exact opposite of caring about security and privacy?

I guess you can choose your provider still? But isn't the point that the lowest bidder is doing inference?

You can set privacy requirements and define an allow list. To me the main value prop is that I get one bill for all models and can quickly try new models without signing up anywhere or changing my code. Oh! Also you can pass an array of models and if the first provider is down it automatically falls through to the next provider. More useful than it should be...

That's still ultimately privacy by contract (where you have to trust the inference providers to uphold their end of the deal), rather than privacy by design.

I always setup guard rails so that only zdr providers are used.

I think the trouble is that Typesafe APIs don't fit into the normal OpenAI-style API that every other regular LLM provider users. You're not just providing unstructured text and getting unstructured text back. It would take a different request and response format than every other model on Open Router. Though you could shoe-horn it in some way, it'd be hacky.

But agreed it'd be very useful to see it deployed on other hubs, and it seems worth it to provide the bespoke API format. Perhaps Typesafe's API will end up becoming the standard for a new type of structured model, the way OpenAI's API did.

It can be shoehorned to work with OpenAI's newer Responses format.

JSON or XML in with json schema for the response?

https://developers.openai.com/api/docs/guides/structured-out...

The thing is, in this climate it's hard to believe such tech will remain secret for long.

So, assuming this is not vaporware, this would raise the tide for everyone because it shows what's possible.

They don't like adding stealth startups :(

Reply on news.ycombinator.com