Comment by tumdum_ | Hacker News Reader

Comment by tumdum_ | Hacker News Reader

I find the attitude shown in this post very surprising. On the one hand, the post starts with a story of adopting Linux and other FOSS. The core of FOSS is giving its users the ability to understand and modify software they run. On the other hand, the rest of the post is about using a tool (LLM) that the author has no way to modify and no way to understand. Huge matrices of floats are at best comparable to compiled code. But the reality is even worse - it’s actually easier to decompile and understand proprietary software. Not to mention the fact the most of the time users can’t even run the “open” models since it requires hardware that most can’t afford.

How did we get from prising software freedoms to this?

I’d disagree wrt “modify”. There are all sorts of tools for modifying LLM weights (ie to remove refusals, remove layers or experts, merge models, finetune, and more) and a quick glance at huggingface or civit will show those in very active use.

I don’t think the hardware requirements are relevant. If a research lab publishes the code their particle collider runs under the GPL, that doesn’t make it not OSS even though they’re the only ones on the planet with the hardware to run it.

You can also edit binary distributions of models with means besides changing their weights. See "LLM Neuroanatomy: How I Topped the LLM Leaderboard Without Changing a Single Weight."

On the spectrum of:

  careful engineering--hacking--mad science

This kind of thing falls far towards the mad science end of the scale, but has proven effective.

https://dnhkng.github.io/posts/rys/

Reply on news.ycombinator.com