Hacker News Reader

Founding AI Engineer at OpenPipe here, using a fine tuned "router LLM" to route between various specialized (inc fine tuned but not necessarily) applied models depending on the input is becoming a common pattern in more modern "graph like" LLM applications.

See LangGraph's "conditional edges" concept here: https://langchain-ai.github.io/langgraph/concepts/low_level/...

You can see how that "routing function" could include a call to a "Router LLM." And yes, fine tuning is a great method to better improve the routing intelligence of said Router LLM.

Great question btw!

Worth mentioning that you don’t even need separate models to implement this. Dynamically loading LoRA adapters is much more efficient, and is the approach Apple took.

Already a big thing. See the constellation architecture used here:

https://arxiv.org/html/2403.13313v1

Very loosely, isn’t this what is happening inside most LLMs that have a “multi-head” mechanism?

Check out https://unify.ai/chat if you're interested in a router optimised for cost/ttft/performance for commercial language models.