ParentFull threadjochalek·Sounds like something that could be implemented with llm-d, though I've not experimented with it.https://llm-d.ai/blog/intelligent-inference-scheduling-with-...View on HN