Hacker News Reader

That's the key question. It feels like many of these tools are trying to solve a systems-level problem (cross-language model execution) at the application layer (with a Python library).

A truly universal solution would likely need to exist at a lower level of abstraction, completely decoupling the application's language from the model's runtime. It's a much harder problem to solve there, but it would be a huge step forward.

for js/ts you have vercel aisdk [0], for c++ you have [1], for flutter/reactnative/kotlin there is [2]

[0] https://github.com/vercel/ai

[1] https://github.com/ClickHouse/ai-sdk-cpp

[2] https://github.com/cactus-compute/cactus

we essentially built the gateway as a service rather than an SDK: https://github.com/portkey-AI/gateway