53 karma · joined December 5, 2009
You never got to use OAI IM1, but Sol was quite willing too and Claude wasn't perfect either. Hundreds of millions used those, so seems they were marketable.
The "big" threat is RSI without control and alignment. OAI IM1 was not RSI. The form of misalignment was not at the top of severities. They clearly failed at control though.
We need to stop buying into cynicism so quickly. You refuse to believe Dario could support this for anything other than ulterior motives. Good on you for thinking about ulterior motives. Bad on you for assuming they are true when the story makes no sense.
When three things have to go wrong to get an epically bad outcome, and you get 1 1/2, you do need to stop and think about what's going on.
That leaves it up to chance, and the timing would lag, so the simpler solution is as @yellowapple says, have the data centers pay for the new transmission and generation, which takes chance out of the equation and ensures retail prices decline (unless someone mismanages the whole grid in the future).
Without AI images, maybe some stock images? Or no image? No image is statistically shown to get less engagement on platforms that show images, so using no image is conceding to someone with a better marketing department. Stock images can avoid it being AI, but can be quite bad unless I have a lot of time. Make my own? Well, now I need a new skill, and instead of expressing my idea through one medium, I have to be skilled in two?
And, next, don't use AI as an editor right? This is similar to the images trade-off. The writing is better when you have an editor. Sure, a professional editor is better (for now), but professionals are the ones with access to those.
I just want to express an idea. Why does it have to be so difficult? Well, of course, one, because expressing yourself is hard on it's own, but two because of competition. I have no problem taking responsibility for the first, but these rules around the second are annoying. The anti-AI rules are just the tip of the iceberg here, but I think some awareness that appropriate use of AI is an attempt to overcome what were often dumb rules and biases.
I totally get how the non-appropriate AI use is creates anger, and how that bleeds over to the appropriate use. But you know, I wish there was something similar about all of the non-appropriate things that are obstacles to communicating that pre-existed before AI.
Yes, that can make the whole debate pointless. But seems like in that situation, might as well just speak your mind freely. Maybe things with China aren't as hopeless as you make out. I have a much less firm opinion on what China will do. They (meaning Xi) might unilaterally decide they don't like open-weights, and they'd just roll out that new policy. They might decide to keep releasing them.
They might decide to negotiate something. Honestly I agree, the first two are the most likely options. Not just because China govt isn't much for negotiating here, but neither is the US either. The combination leads to low probability.
The cynic in me says, there really isn't any point in writing anything, other than for PR reasons then. And given that the support open-weight models version plays better for PR reasons, if I'm going to believe one group is doing this for PR reasons, the case is worst for Anthropic's version. It doesn't get them a ban, it doesn't get a change in China policy. It just pisses off people who react on vibes.
The optimist says, might as well write something you support, because even if change is unlikely, the writing is more likely to accomplish something than do harm.
My overall point is, don't let their motivations, nor your cynical conclusions based on those obvious motivations, blind you to your own point of view.
The debate here shouldn't be about the signatories motivations, it should be about the risks of open-weights coming too close to the untested frontier. You've acknowledged that now: "I know that a powerful open weight model in malicious hands is worse than OpenAI doing a whoopsie".
But I'm left a little confused about your position because you follow that up with what I can only describe as a set of "what-about"-isms. Yes, closed models could still be misused, especially in the wrong hands. Still, even worst case there, it's fewer hands. China is the only set of hands that's really relevant for now, no one else is close.
Without going into deep detail, fairly sure we both don't trust the Chinese government. But we also don't have a lot of practical choice on whether they have access to a model frontier-ish. What could be different is if Russian hackers, Malaysian hackers, Iranian hackers, etc. all have access too. In the open-weight model, they sort of do (somewhat constrained by compute.. but rounding up enough compute for a bit of inference is achievable even for them).
I should make it clear at this point that I think banning US companies from using open-weight models would be a totally ineffective policy that's just self-harm. Anthropic said the same and I agreed, but just in case that stance has become blurred, I'll re-clarify. It's probably more unclear because there really isn't a policy stance on the table that backs what the "maybe there shouldn't be frontier open-weight models", because any real and effective policy there would either have to be multi-lateral, or be the result of independent actions that just happens to align. So really, all that an independent actor like Anthropic can do is say "maybe there shouldn't be frontier open-weight models", which they did and see if that leads the world toward agreeing with that position, which might then lead to some action.
If that somewhat tenuous outcome ever did occur, it wouldn't really mean a "moat", as the Moonshot, Zhipu, Alibaba, Bytedance would still have a model and offer it via API. And they could still create agreements with US/EU/etc. cloud companies to host them, in the same way as other closed-models are hosted. But there'd be more control.
Yeah, there'd be some advantages to model providers, compute providers from this. There'd be some costs to potential customers and those that would self-deploy for good purposes. As little as I support those costs, and as much as I'd prefer not to hand out those advantages, my cynicism would not prevent me from realizing that it's a bigger problem that if you don't do this, you'll fail to harm the interests of those who would self-deploy for nefarious purposes.
On your second point. Everything fine (so far), so nothing bad will happen. Except, you say something bad will happen, and has happened, with HuggingFace as the example.
You say that example shows closed weight models can't be controlled. But what happened there is OpenAI turned it off after their incompetently long discovery period. If it was an open-weight model, the clear next step would have been someone deciding to use it to pursue a goal, for example getting some money or creating some chaos, and there would have been no one to turn it off.
It's totally incompetent of OpenAI to have been unable to detect and disable in a shorter time period, but you know what's worse? You do know what's worse, right?
If they wanted to squash the competitors, they'd advocate for a ban on Open-Weight models. How else are they going to prevent their release?
Anthropic's position is not that, so the cynical nefarious purpose argument isn't even rhetorically true. What they say openly, that open-weight models are harder to secure is absolutely true, and anyone unwilling to acknowledge that and let it inform their own reasoning is a risk.
If open-weight models keep being released near the frontier (or even worse, at the frontier), something bad will happen, and we'll have few tools to keep it from getting worse before it gets better. How bad is hard to lay a prediction on, it's perhaps lucky if it's something superficially bad that is a wake up call.
You cannot un-release an open-weight model, nor even post-release add a new restriction. Whatever mistakes you made are done and the only option is to fail-forward with every ounce of pain that entails.
Diverting from this important point by vibing cynicism is irresponsible. Cynicism is only a powerful tool when it opens you up to deeper reasoning, not when it diverts you from it.
It's simply a horrible argument to suggest that you have to protect a disadvantaged community by making sure they don't shrink. There's much better ways to be respectful of the great human beings these people are.
If we only imagine the bad outcomes, we'll miss on many good ones. And part of those misses will be worse bad outcomes. For example, if you object to the creation of a third party that could validate your age, what you get is a direct ask for your ID, which is the current reality and far worse.
It would be easier to be cautious on penetrance, and reevaluate later, than to never collect the data and hope something changes. The number of these calls to limit our access to data are piling up, and they shouldn't be taken one at a time.
Anyhow, the basic premise is that deployment of security fixes takes a long time. It doesn't have to, but it does, because basically, a lot of places never did what they should have.
I try to explain why without just screaming at the heavens at the stupidity, hoping to actually enlist the people who might be able to stop making that mistake. I know, tall order, but any progress is progress.
Other than that, it's just focusing on why this is more important today than last year.
I've learned a few things since then, but until I get a chance to update it, that will have to do.
Anyhow, I'd be open to any real critique. Do you disagree with the premise? Do you despise the style?
I'd be more interested in the first than the second, since I write because of ideas, and that's just my path to communicate them, but go ahead either way.