Comment by wxw | Hacker News Reader

Comment by wxw | Hacker News Reader

> Inkling is not the strongest overall model available today, open or closed. Instead, a combination of qualities makes it a good open-weights base for customization: multimodal capabilities, efficient thinking, and availability on Tinker for fine-tuning.

Open base models that can be fine tuned on Tinker is a great business model IMO. You (i.e. an enterprise) can own your own model & have it perform frontier-or-better at your task at potentially much lower cost and Thinking Machines gets to be your essential infra/service provider in this world.

Also,

> Inkling-Small matches or exceeds its larger sibling on many benchmarks — the result of improvements we made to the pre-training data and recipe for the smaller model.

Very cool! Excited to see the next generations of Thinky models.

> that can be fine tuned on Tinker

Good source to understand why this is valuable?

Frontier models need to do everything for everyone. It's expected (though not often done) that smaller models fine-tuned on specific tasks can approach frontier performance on a specific area. [0]

Post-training/fine-tuning is not trivial and having it as a service might make it more accessible.

[0] https://surgehq.ai/blog/training-on-complexconstraints

If you want an LLM to have knowledge about something, the knowledge has to either exist in its weights or be provided to it in its context. Because context is expensive and limited, and models tend to get dumber the more their context is filled, there is usually more that you'd need to put into context than can reasonably fit in it in order for the model to answer questions about your data. So your options are basically

1.) stuff it into context

2.) figure out a way to determine what to put into context based off of what is being asked of the model (RAG)

3.) change the weights of the model to have knowledge of your data baked into it (fine tuning)

Do you think 3 is better than 1 & 2 as context gets larger? I think for smaller data sets its mostly fine no. It's an interesting bet by TM. End state does seem some form of continual learning (model weights update like dreaming)

Reply on news.ycombinator.com