Hacker News Reader

Comment by geoduck14 | Hacker News Reader

I fine-tuned an LLM to do technical stuff. It works pretty darn good. What I actually discovered is that when evaluating LLMs, it is surprisingly difficult to evaluate them. And, also, that GPT 4 isn't that great, in general.

View on HN

Could you provide more details on this matter? Specifically, I'm interested in knowing which base model you've utilized and the approach you've taken to fine-tune it. Your insights would be greatly appreciated and highly beneficial.

For narrow stuff you can do better job than base gpt4/mistral/etc model. You fine tune it with your very custom data, stuff that got didn’t seem to be trained on, it will generalize it well.

Have you done this? How did you do it?

I've been looking forward to someone providing a detailed guide on how to "fine tune it with your custom data" for ages!

this is imo the secret sauce that gives people an edge and not a lot of people will want to reveal

You're not wrong. There's been a lot of drama over licensing and releasing datasets, and a lot of the LLM scene are just pitchmen and promoters with no better grasp over what they're doing than "trust me, it's better".

Like with "prompt engineering", a lot of people are just hiding how much of the heavy lifting is from base models and a fluke of the merge. The past few "secret" set leaks were low/no delta diffs to common releases.

I said it a year ago, but if we want to wowed, make this a job for MLIS holders and references librarians. Without thorough, thoughtful curation, these things are just toys in the wrong hands.

https://www.datacamp.com/tutorial/fine-tuning-llama-2

This is a very nice resource: https://github.com/mlabonne/llm-course

Maybe the key to a good universal LLM is having multiple fine tuned models for various domains. The user thinks he's querying a single model but really there's some mechanism that selecting the best model for his query out of say like 300 different possibilities.

This also helps distributes traffic as a side effect.

I guess the problem is how the conversation would flow. If the user changes topics from say art to quantum physics then asks a question about quantum physics and art then I'm not sure what the algorithm should do.

The user could talk to an "expert opinion aggregator" model which in turn makes a bunch of queries to specialized models.

That is actually the same idea as the (now) popular "Mixture of Experts" approach.

> This also helps distributes traffic as a side effect.

I'm not sure it's "distributing" traffic so much as amplifying it.

Two users. One user is talking about physics, the other about art. Two different models are utilized.

Load is divided across 2 models. Load balancing is a feature for free and division is across subjects. Of course this is assuming each model owns it's own set of gpus.

You could use the same GPUs to run two instances of one model.

Ah okay, I misunderstood. I thought you meant inferencing on all the different models and letting them judge to what extent they're the best fit.

What you're suggesting is just simply intent classification and using a specific model per intent. That's what everyone did _before_ LLMs.

Given the existence of specialized LLMs people can still do this.

> GPT 4 isn't that great, in general

same here, it doesn't adhere to explicit instructions, maybe one or two simple instructions are ok but not more complex ones

YMMV, it depends very heavily on what you're asking it to do.

More complex instructions should be split into multiple prompts, solving the challenge step by step

Done this way, you also re-incur any input token costs for every additional step. If you're using a large context, this can be significant.

Yes, but sometimes the two parts are also correlated so it's better if they are done at the same time.

What model are you using for more complex prompts?

Which model did you start with?

Llama2 Code

Which base model did you use and what methodologies you used?

Likely mistral

Reply on news.ycombinator.com