> open weights, open code and open data
Even if you have all these things you still can't replicate a model because of randomness.
You can backdoor a model with less than 1000 examples and it is impossible to detect.
You can take the code for Kimi K3 now, take the training framework from Prime and the data from Olmo, spend some money on RL environments and some more money (!) on GPU training and end up with a system of similar capabilities.
But that's completely different to being able to audit Kimi K3. Even if you had the exact code, data and training environments it is impossible to verify that the model you have came from that.
They just don't work at all on a many month long, 100K+ GPU cluster training run.
Even then you'd still need to account for order of events when an entire cluster of GPUs is involved. Also don't forget to account for any synthetic data sources. Or even non-synthetic for that matter - does your pipeline do any image resizing on the fly? Better make sure that's fully deterministic between machines (it almost certainly won't be).
It's theoretically possible but I don't expect it to materialize any time soon.
I mean I guess, but not in a performant way if there are ever any hardware failures. And with 100K GPUs there are multiple hardware failures per day.
Where is this meme coming from? IEEE floating point is deterministic
The context (verbatim):
> Correct. We need open weights, open code and open data. If nobody else can reproduce what someone did there will always be security questions. Even if we can reproduce it there could still be security concerns but it's more realistic to investigate yourself.
In short, it's an appeal to full openness and reproducibility on the basis of security; open weights alone notably do not provide that same confidence. They're better in some respects, not really in others.
Then comes the question (also verbatim):
> Exactly what are the possible 'security issues' of self hosting an open weights model?
Implying then that as long as you do have the weights and just self host it, the asker cannot imagine what could possibly go wrong. What is the gap, if any?
And so I explained. That was my point. Open weights do not give you full reproducibility, and so that on its own falls short of what the parent comment is making an appeal to. That there does remain a security concern, shared by remote and closed models, that does not improve just by having the weights, but would if you did have full reproducibility. Explaining that gap was my point, as that is what I understood as being asked there. It's the only thing I can reasonably imagine being asked, in fact.
This is a materially different question to what you apparently extracted (again, verbatim):
> What security issues come from self-hosting?
Implying that by self-hosting models, something bad might specifically happen.
I do not think this, do not think I suggested this, do not think the original question suggested this, and generally do not think this is indeed any sensible, in or outside the context. Certainly not beyond something common sense, like vLLM being compromised or whatever.
You seem to agree. But then how did we get here, clearly talking past each other?
Even with this, the cost of verification would be enormous. You would need a massive cluster to repeat the training E2E.