Show HN: LLMSim – a fast OpenAI LLM API simulator for load-testing LLM apps

Testing LLM apps and agent frameworks against real APIs is expensive, rate-limited, slow, and non-reproducible. LLMSim is a Rust simulator for the OpenAI Chat Completions, OpenResponses and in future other protocols.

LLMSim could be used in two forms, one is the server, with high concurrency for load/stress testing (~40k req/s with p99 ≈ 5ms on 4 vCPUs, scaling with cores); and second it embeds directly as a crate in your tests - no separate process, no network, deterministic.

It focuses on traffic realism, not answer quality: configurable time-to-first-token and inter-token latency (normal distributions, per-model GPT/Claude/Gemini profiles), SSE streaming, real tiktoken token counting, and error injection (429/5xx/timeouts) for chaos testing.

Usage is super simple,

cargo install llmsim --features tui

llmsim serve --port 8080 --host 0.0.0.0 --generator lorem --target-tokens 150 --tui

And then any LLM client `http://localhost:8080/openai/v1`

MIT