LLM Fine Tuning Guide for Enterprises in 2023
research.aimultiple.com
research.aimultiple.com
1. The article doesn't mention the downsides to fine tuning.
The more you fine tune an LLM the more it loses its generic abilities. Sure, it might do a bit better with some domain-specific terms, but in exchange, it's likely going to do a lot worse in unrelated domains. This might be ok, but it might be terrible. Depends on what you're doing.
2. It's technical nonsense to promise that fine-tuning an LLM will make it comply with regulations and keep sensitive data safe.
> Businesses handling sensitive data or operating under strict regulatory environments might need to fine-tune the model to ensure it respects privacy requirements, adheres to content guidelines, and generates appropriate responses that comply with industry regulations.
Telling people this is actively harmful. Absolutely nothing you do with fine-tuning can ever guarantee that an LLM will respect privacy, that it will adhere to content guidelines, or that it will generate appropriate responses. Fine-tuning is the wrong tool here.
3. The article conflates domain-specific and task-specific fine-tuning. These two are totally different ballgames: do you want a network that's far better at parsing or do you want a network that knows about your domain?
4. The article doesn't talk about how fine-tuning can fail you. How it can be hard. How setting up the data can be tricky.
I have no idea who Cem Dilmegani, he's certainly not an ML or NLP researcher, but from this article it's clear he has no idea how LLMs work and has no business trying to explain anything about AI or ML to anyone.
Any better resources on fine-tuning LLMs?
- Synergizing Reasoning and Acting in Language Models: https://react-lm.github.io/
- Augmenting LLMs with Knowledge: https://www.arxiv-vanity.com/papers/2306.08302/
- Grounding LLMs with Retrieval-Augmented Generation: https://techcommunity.microsoft.com/t5/fasttrack-for-azure/g...
The robot knows a little about a lot of things, but the extent of its knowledge about any specific thing is generally limited.
Fine-tuning (or just tuning) basically equates to some extra training to become better at some specific task. In a very basic sense, fine-tuning a model with additional data about some task just trains the model to know more about that specific task.
For example, you might buy a general-purpose cleaning robot for the home which can do a lot of different cleaning chores, but if you have special china plates that need to be cleaned in a certain way, you would want to train your robot a little more than its factory defaults. That extra training is "fine-tuning".
How/why/where do you know how to 'tune' the robot? Vant we have robots training other robots on specific things? Can we have a robot specific to chemistry - then have deep diving robots on specifics of a particular element, then informing the parent AI with the details a specific LLM(?) has known...
(apologies if my ignorance hurts you... I dont really know where/how to come up to speed on all this AI BS)
What do?
What would actually be useful would be showing a specific, concrete, non-trivial example of fine-tuning an open source model.
The other info about the ecosystem is useful, but actually show the work rather than sell other companies products.