Comment by cowl | Hacker News Reader

Comment by cowl | Hacker News Reader

Parent Full thread

yes there is. how LLMs "learn" is by training and every new "fact" moves the weights of older learnings. you can not be sure that this movement has not caused past learnings to be forgotten or brought to absurd limits without a full cross training. that is why everyone does training in stages and doesn't just let the models learn gradually daily. for LLMs all "knowledge" is in one big container to be updated all or nothing. Humans on the other hand, learn in different containers. you have your core beliefs that are not touched everytime you read or hear something new, it requires a pretty good shake to make anyone "update" their core beliefs. from there there are several corpuses of knowledge more or less isolated from one-another that may take various amount of "influence" to change but that more or less don't impact each other. for example learning a new foreign vocabulary does not really impact your math knowledge etc.

notice that LLMs "context" chat is not learning, it's a temporary effect that gets lost as soon as the chat closes.

What about fine-tuning?

fine-tuning is not learning, it's controlling the response and you see it's absurd effects in countless examples where in the name of being politically correct the "weights" have been modified also for the past. (classic example of Gemini's German Nazi "representative" photo or even more: https://art-for-a-change.com/blog/2024/02/gemini-artificial-...)

The mechanism for fine-tuning and the original training is exactly the same (gradient descent on weights). The effects you describe are results of what exactly is used to fine tune on.

the mechanism is the same (that's why it impacts all weights) but the target of gradient descent is not the same. in finetuning they aren't saying "go down the mountain" anymore, but "go down toward this plateau" ofcourse this changes the gradient.

is it "learning" in a certain sense? sure, the same way like all indoctrinations are sold as "teaching". the model "learned" to be rapresentative but forgot what 1940 nazi soldier was like...

and fine tuning is not a scalable approach because it has human feeback in the loop. could they fix this error with more fine tuning? yes and they tried abut then the users simply asked "give me a picture a viking warriors" and the problems was again there, you can't fine-tune everything even if we assume the purpose is always noble.

I think all this ideological stuff is completely unrelated to the issue we started talking about. You can fine tune a large model on whatever data you want, and so you can also fine tune it on the most recent user inputs. You can do unsupervised fine-tuning btw, no need for human in the loop. It all depends on what you want to achieve.

Reply on news.ycombinator.com