Hacker News Reader

You have a restaurant and have 2 types of chef.

One is the sous chef , who knows recipes and cooks fast, handle most dishes as quick as possible. He does not think much. That is your medium effort llm in charge.

One is the head chef, who is inspecting, pause, consider edge case. Slower but catches what sous chef missed. That is your high effort llm in work.

View on HN

But I was thinking Chef is the LLM model. So I was wondering if it is the same model, what actually changes the effort? Is that part of the inference?

So are you saying that effort level isn't a parameter to an LLM, but that each level actually corresponds to a slightly differently tuned LLM?

I think the analogy here is time. Especifically performance time. Head chef performs slower. High effort is longer inference time. For LLMs, the ability to think in more detail requires more time. Maybe this will change in the future but it is the present now.

Reply on news.ycombinator.com