ArchGW: Open-source, AI-native (edge and LLM) proxy for prompt traffic

Why We Built ArchGW?

Traditional application architectures separate routing, security, and observability from business logic - so that developers can move faster without the tax of reinventing the wheel. LLM applications should be no different. ArchGW applies these patterns to prompts, providing a structured approach to building LLM applications.

How It Works

ArchGW runs as a separate process alongside application servers. It intercepts prompts before they reach the backend and applies transformations based on predefined rules and models:

    Preprocessing: Normalizes and analyzes prompt structure.
    Security Filtering: Rejects jailbreak attempts and unsafe inputs.
    Intent Mapping: Determines if a request maps to an API function.
    Function Invocation: Extracts arguments and calls backend APIs.
    LLM Routing: Chooses the right LLM provider based on latency/cost constraints.
    Tracing & Metrics: Adds W3C Trace Context headers, tracks errors, token usage, and request latency.

Why a Dedicated Proxy?

Traditional application architectures separate routing, security, and observability from business logic—LLM applications should be no different. ArchGW applies these patterns to prompt processing, providing a structured approach to LLM integration.