AMD Unveils Its First Small Language Model AMD-135M

_Always_, with the right options. And that's the key point. If distributed code is different it means it may be infected or altered in other way. In other words it cannot be trusted.

The same with models, if they are not reproducible or verifiable they cannot be trusted. Trust is the main feature of open source. Calling black box with attached data 'open source', even 'the first' is a bit of a stretch. It's not reproducible and not verifiable. And it's definitely not the first model with open data.

To be correct you should add 'untrusted' if you want to call this thing 'open source'. Like with Meta's models who knows what it holds.

PS: finally I'm negative, fanboys don't like it ;-)

https://opensource.org/osd

Personally, I am all in favor of increasing commercialisation of academic research, if and only if the money earned goes first and foremost to reducing tuition for students.

Alas, tuition seems to ever inflate upwards. I remember when my university built a football stadium- all sorts of talking points were floated about, like how the sports programs pay for themselves, it's good for recruiting, we wouldn't get increased tuition, blah blah blah. Well, once construction finished, guess what? Students got a $400 stadium "fee" added onto enrollment each semester. Sure, it wasn't technically part of tuition, but it also wasn't optional.

That alone pretty much guaranteed that I ignored every attempt from the alumni association that came begging for donations after I graduated. They run a well oiled machine, and yet somehow no matter how much money the school had, students kept paying more each year.

So, yeah, I do know many (most?) research schools have commercializing offices dedicated to making money off of their research, and they have every incentive to keep doing so, whether via parents or keeping things secret to commercial partners. They just don't do anything publicly visible with the money.

Edit: sorry for the stream of consciousnesses, I'm running on an hour of sleep and just realized I don't have the energy for a coherent, concise response

> Can I build this thing from scratch myself

You absolutely can. Everything you need is in the model config (layers, stuff) and there are training scripts all over the net. Now, granted, you will not necessarily get the same results, but then again neither is Mistral or Meta.

> but the authors tries to argue that the compiled asm output is "open source". It's output, not source, so you cannot license the output as "open source" as it's missing that last part, the "source".

Replying here because I can't in the other subthread. I think you are using a misconception on what is source code, and what is a weight. In the LLM world, you already have the source code for inferencing. This would be either pytorch or c code or whatever. You also have the architecture code. You can see what the model looks like, what layers it has, what ops it does to reach a result. That is also open! So you get the source to run inference. You get the source to "load" the model (i.e. the architecture, layers, etc). And you get a bunch of hardcoded values.

What you don't get is the why behind the question "why is this value x and not y". And for the most part, no one knows.

> If they're not available, or available but under restrictions (usage or otherwise), then it's not open source.

Let's take another (famous) example. Quake is famous for having a hardcoded value somewhere in the source code, that speeds up some geometry computations. Now, you can change that value, but things will be messed up in the engine. Collisions will happen weirdly, things will look bad. Now, is quake any less of "open source" if you or I don't understand why the original coder chose that value? Of course not! Well, now just multiply that with 1B hardcoded values. It's the exact same thing. You could change any of the values, but the game would look wonky as you do so. But, at the end of the day, it would not be any less open source.

I guess what I'm trying to say is that weights are not binary blobs. Weights are not an obfuscation attempt. Weights are distributed exactly how they are intended to be used, and how they are being used by the creators as well. You can change the architecture of a model (see above for details). You can add layers, you can remove layers. You can perform "abliterations", or you can do fine-tuning. Everything is exactly done as the "creators" intended. The only thing you don't have is "how they got those exact same numbers". But you don't need that. And it's funny that somehow for LLMs that's a bridge too far. It never used to be for any other project.

People view weights as an intended obfuscation by the party releasing it. It is not! In fact, it is equally as hard for them to "understand" why a certain value at a certain index is what it is, as it is for you! Just ask Anthropic. They are also doing poke this weight, see what pops with their own models.

Again, that is why I used the analogy above. You are arguing that if someone uses a hardcoded value in their code, and won't share how they derived that value, it somehow isn't open source. That, IMO, is wrong.