Hacker News Reader

Hacker News

Comment by ComputerGuru | Hacker News Reader

Parent Full thread

ComputerGuru

They “open sourced” it enough (via the whitepaper) that huggingface is trying to reproduce their training now.

View on HN

How is it open source at all with no source?

Paxos isn’t open source just because you can read the paxos paper.

They released the weights

As an actual FOSS developer: they didn’t open source it.

But I was merely adding the missing context using the (sorry) lingua Franca of AI.

For people who have the disciplinary background in neural networks and machine learning I imagine that replicating that paper into some type of framework would be straight forward right? Or am I mistaken?

The model itself yes. The changes from previous architectures are often quite small code-wise. Quite often just adding/changing few lines in a torch model.

Things like tweaking all the hyperparameters to make the training process actually work may be more tricky though.

With an LLM, the actual 0s and 1s of the model are fairly standard, common, and freely available to anyone that wants to use them. The "source code" for an LLM, is the process used to create the outcome, and to an extent, the data used to train with. DeepSeek released a highly detailed paper that describes the process used to create the outcome. People/Companies are actively trying to reproduce the work of DeepSeek to confirm the findings.

It's more akin to scientific research where everyone is using the same molecules, but depending on the process you put the molecules through, you get a different outcome.

> With an LLM, the actual 0s and 1s of the model are fairly standard, common, and freely available to anyone that wants to use them

How is that different than the 0s and 1s of a program?

Assembly instructions are literally standard. What’s more, if said program uses something like Java, the byte code is even _more_ understandable. So much so that there is an ecosystem of Java decompilers.

Binary files are not the “source” in question when talking about “open source”

There is no way to decompile an LLM's weights and obtain a somewhat meaningful, reproducible source, like with a program binary as you say. In fact, if we were to compare both in this way that would make a program binary more "open source".

Yes, that is my exact argument.

Publishing a white paper doesn't qualify as open source in any other context.

Google Spanner has a nice white paper but you wouldn't consider it open source, for example.

Reply on news.ycombinator.com