Hacker News Reader

Comment by markanton | Hacker News Reader

Nice project but there are several dozens of “AI/LLM gateways” now.. all kind doing the same thing. Kong AI gateway [1] was maybe the first to attack the LLM traffic governance and is indeed far ahead in both features and adoption. Trying to understand the value add and differentiator here, since it’s a problem kinda solved already.

[https://github.com/Kong/kong]

View on HN

There are a few critical differences. archgw is designed as a data plane for agents - handling and processing ingress and egress (prompt) traffic to/from agents. Unlike frameworks or libraries, it runs as a single process that includes edge functionality and task-specific LLMs, tightly integrated to reduce latency and complexity.

Second, it’s about where the project is headed. Because archgw is built as a proxy server for agents, it’s designed to support emerging low-level protocols like A2A and MCP in a consistent, unified way—so developers can focus purely on high-level agent logic. This borrows from the same design decision that made Envoy successful for microservices: offload infrastructure concerns to a specialized layer, and keep application code clean. In our next big release, you will be able to run archgw as a sidecar proxy for improved orchestration and observability of agents. Something that other projects just won't be able to do.

Kong was designed for APIs. Envoy was built for microservices. Arch is built for agents.

fwiw, if I were evaluating these proxies against each other, I would be intrigued by the solution built by people from the Envoy team. Envoy is great software and I’m sure there are many lessons you took from building it.

It looks like you’re even building on Envoy as the foundation for the system which just makes it more compelling.

Its a core dependency for rate limiting, traffic shaping, fail over detection. Its cluster subsystem is super convenient for local LLM calls too. We'll write up a blog on the lessons because there were many. For example, for intelligent routing decisions we can't create an upstream connection to a cluster based on route paths or host - Envoy forces a more static binding. This doesn't work when you are making decisions about a prompt and have to inject more dynamic flow control.

MCP is simply an API protocol, like GraphQL or gRPC.

And since everything is an API, Kong also supports MCP natively (among many other protocols, including all LLMs): https://konghq.com/blog/product-releases/securing-observing-...

MCP implementation is trivial - I agree. But A2A will require a mesh like structure. Meaning its not just about north/south traffic. It will be about east/west traffic as agents coordinate with each other. That communication and coordination among agents will need to be robust and that's where a sidecar proxy built on top of Envoy will offer certain properties in a first-class way that Kong can't easily support today.

This was the insight behind Envoy's initial design. Handle north/south and east/west traffic equally well as a universal data plane.

Reply on news.ycombinator.com