Comment by runeks | Hacker News Reader

Comment by runeks | Hacker News Reader

Somewhat off topic, but I've started wondering if we can actually make local LLMs feel smarter than the frontier closed models, by post-training it for your specific use case.

Say Company X has a software product which consists of a million lines of code, including a ticket for every bug and new feature for this piece of software. Then wouldn't it make sense to try using an open weight model but post-train it on that specific code base while using the tickets to teach the model about past bugs and features. Not sure exactly how, but it could involve doing reinforcement learning solving a past bug on that historic version of the code base, and rewarding the model if it comes up with the correct solution (as defined by the linked PR which fixed the bug).

So essentially post-training your local open-weight model using reinforcement Learning with Verifiable Rewards (RLVR) on your software products history of bug reports and their ultimate solution. And the same for new features.

You could do this for sure but its quite a lot of effort and a normal software stack is quite well known due to the massive amount of github projects and other opensource code.

You can also get a lot out of a harness in this case or your Agents.md or Claude.md file by just enhancing the context.

You might even question yourself if you are doing something wrong if a modern LLM really struggles with your code.

We do the finetuning only on small semantic data were it helps a lot.

I'm still wondering when we see smaller models (faster and cheaper) for more specific stacks like spring boot + java + angular + english only or so. Interstingly enough, i assumed LLMs are really good in any language but it seems that non english languages do reduce the ooutput quality of an LLM. At least last years GTC there was a talk about it.

There are companies though which ahve this exact problem with programming languages you normally don't see. ABAP for example is a very well known SAP language.

I wonder the same thing. Code is pretty open ended though so I wonder if it’s not the best example.

On the one hand it would definitely be useful for something like classifying support requests into priority. But would it be worth it to just train your own model? I guess one advantage is you could give it some well informed guidelines without training something on lots of data.

There is probably a better example between discrete labeling and code though.

Instead, I'd build tooling for the model to be able to query the tickets, pr, commits, diff, etc

This is something I can reuse better.

Reply on news.ycombinator.com