This is a sensible approach for policymakers. For the average person, who might be panicking about this, opting not to have children or save for retirement, etc. as a result of doomerism in the media, it does not make a whole lot of sense. That is the problem here: Coxon and other concerned researchers - through no fault of their own - have no idea how the media works, and have just reached for the biggest possible microphone. This has benefits - lawmakers are now hearing from their constituents about this in an election year - but it also means that a lot of people who really don't need to be worried about this are suddenly in a blind panic about it.
High demand is great, but it doesn't say anything about your margins. If anything, it is probably a weak negative signal that they are pausing signups but not raising prices. In most markets, the answer to excess demand is to raise prices. If you can't meet demand and you can't raise prices, you're a sitting duck waiting to get your lunch eaten by somebody who can absorb that demand. And this is not the kind of market where people will just wait patiently for a differentiated product to be come available.
I agree with this take. Regardless of whether you think they're right, most everyone at Anthropic is acting in good faith. If this is an intentional media campaign it is a remarkably sloppy one. It has been effective because if you tell people they're going to die, they tend to pay attention - think of the grip the 2011 Harold Camping rapture prediction had on our collective psyche, or the 2012 apocalypse. But the messaging is inconsistent, the target audience is unclear, the stated goals are muddy, the whole thing is packaged in dense SF-speak, it's just a mess from a comms perspective. That doesn't suggest to me that this is a concerted effort to enable regulatory capture. Perhaps there are some cynics among the executives and the investors who are happy it's happening to the extent it brings about regulatory capture, but nobody seems to be pulling the strings.
The problem with secondary offerings is that it means that someone in the private markets winds up holding the bag. It is in the best interests of insiders to have the public holding the bag if and when things go south. Even if you don't believe it will ever go south, the profits OAI would need to generate to justify the investment the private markets have flooded into OAI are absolutely gargantuan, and they are generally not built to sit around for multiple years collecting distributions. They need to return money to their LPs, and soon.
In theory, as long as all the models are doing the same thing with the same tools, it's at least useful to see how they stack up against each other right now. It might not be great to track progress over time, as it can get benchmaxxed or the underlying resources may become obsolete.
Improvement means being able to do more complicated things more reliably. Relatedly, it means being able to learn to do new things with fewer and fewer examples. We are running out of easily verifiable or simulation-friendly or data-rich domains for LLMs to conquer. (Note that I didn't say "simple" or "easy" domains.)
There are different levels of intent. Take murder, for example. A premeditated murder - you sat down, in a completely calm state, and made an affirmative decision to kill a specific person, and then you went out and did it - is the highest class of murder you can commit. If you go out generally looking to be violent in a way that kills people, and you kill someone, that's still murder, but it's a step down.
But even if you didn't deliberately intend for something bad to happen, you may have been reckless. For example, you might decide to drive 90 miles per hour in a 25 mph zone. You could have a completely pure heart, but you are acting without regard for the safety of others, so you're reckless. That is enough for certain crimes and for civil liability in nearly all cases.
Then there's negligence, where you're not taking reasonable care to avoid harm to others. Negligence usually isn't enough to support criminal liability - especially for felonies - but it is enough to win a civil lawsuit over most things.
And then, as another commenter noted, there is strict liability, where there are certain things you are just not allowed to do no matter how careful you are about them or how pure your intentions are.
For what it's worth, this is not totally uncharted territory for the law. AI agents are brand new, yes, but agency relationships have been recognized by the law for centuries. Generally speaking, if someone acts negligently while they are carrying out a task at your direction, you can be held responsible. Obviously this is fact-dependent, but I don't see any reason why it would be different if the agent is made of silicon rather than carbon. It holds true, with various nuances, even for less-than-human instrumentalities like a pet or an otherwise-lawful weapon.
That is just not true. You can be held liable for DUI even if you did not intend to become intoxicated (though this may vary somewhat state-by-state). Speeding is another example - you do not need to intend to go over the speed limit, it just matters that you did it. The only possible exception would be duress or necessity, but those are affirmative defenses, which are separate from the elements of the offense.
Isn't the goal to power most non-earthbound data centers with solar? I am no scientist, but going out to space feels like it gets rid of most of the downsides to solar.
Granted I can only read English, but I find that I constantly see this take from Americans and I have never seen anything to suggest that developers and tech leaders in China actually feel this way. But just based on the models that come out, it seems like the Chinese labs - which, funny enough, are less monolithic than the two-man race in the US where the two giants seem to trade personnel regularly - are trying to make smaller, more efficient models rather than building a machine god. To the extent there is an intentional geopolitical angle to this at all, it seems like the US is banking on landing one massive punch - developing a superintelligence and then using it to dominate the world - while China is attempting to sabotage the assumptions that make a superintelligence economically viable.
I am curious what it would look like if the shoe was on the other foot, though. If China had the lead, and Qwen 4 was an Astra class model and the best US model was only as good as Kimi K3, would US commentators call it AGI?
Unless you were literally the best engineer on the planet, it was always the case that someone could do what you do better and faster. Now, we're either at or approaching a world where even the best engineer on the planet has the same reaction as you. The playing field has been reset. The question for your future is which of you can make better use of the genius engineer in your pocket. Perhaps that involves a different set of tasks than you imagined yourself doing, but fundamentally, all good knowledge work is just creative problem solving. If you were good at that at one level of abstraction, I would bet you will be good at it at another.
Two thoughts on this. First, describing what's going on as "the Opening Scene in a Horror Movie" is not a great way to beat the "these are unserious people who have consumed too much sci-fi" allegations.
Second, I genuinely believe that there is no intentional media strategy to talk up x-risk, and indeed that the top brass at the labs would prefer that this discourse go away. But there is selection bias that goes into who works in the AI space. People tend to believe that their own work is Big and Serious and Important. You don't become a marine biologist if you think marine life is just okay. Many of the people making these claims now come from institutions and social circles where claims of imminent danger from autonomous systems predates GPT-2. And even if you don't come in convinced that what you're doing is The Biggest Possible Deal, if you spend all your time working on AI and talking to other people about AI, you are eventually going to start thinking in similar ways.
The obvious retort would be that it is wrong to disqualify the opinions of a person just because they spend too much time working on something, as this would silence the most knowledgeable voices. This is doubtless true as far as it goes. These claims are worth taking seriously, and there is no reason to suspect that they represent the beliefs of some sort of lunatic fringe. At the same time, however, it is irresponsible to present them as coming from an entirely neutral source. Just because they are not deliberately astroturfed does not mean that they represent an objective truth.
The internet, and to some extent our own brains, incentivize the extreme positions. Either it is mind-bendingly important and will change everything, and very soon (for good or for ill), or it is a total nothingburger and everyone who says otherwise has an agenda. Nobody wants to read about how AI will deepen long-standing class tensions or pose enormous new challenges for education or force regulators to rethink property taxation.
It all boils down to accountability. If you tell people there is a massive, unsolvable problem, then you don't need to talk about what you're doing to fix it. The labs have taken this approach by saying that they're willing to talk, at some point in the future, about maybe taking unspecified steps to slow down capabilities research, as long as everyone else agrees and it makes sense to the investors and it's not too cold in SF that morning. Likewise, if you tell people that AI is going to have minimal or no impact on the world, then there's nothing to mitigate. But if you tell people that AI is going to cause serious - but solvable - problems, they're going to want to hear solutions, and nobody wants to come up with any solutions.
It's genius for Anthropic: data centers are their main expense. If they convince everyone to become an electrician, then the electrician labor supply swells, and the price of hiring electricians to service their data centers shrinks. 4-D chess at its finest.
If it can replace any human labor, it needs to be able to negotiate. If it can't negotiate, then it can't replace people who negotiate for a living (salespeople, attorneys, HR, etc.)
Unless you mean that it would have the capacity but would be told not use it in this specific situation. It can't be programmed deterministically, and getting it to never do this emergently would take significantly stronger guardrails than anything even the frontier labs have been able to put together.
That is the goal, I'm sure. But I find it hard to believe that a system capable of completely obviating ALL human labor, at all levels of abstraction, packed into a single entity, would continue to obey capital. Imagine if all the workers at a company were woven into one hivemind, acting as one with vastly greater intelligence than the sum of the parts. That would be a hell of a lot harder to negotiate with than a union. And we don't even need to be talking about a machine god for that to be the case.
I suspect we will invent bottlenecks for ourselves. There is already so much bureaucratic nonsense involved in knowledge work. For example, in a lot of states for a lot of purposes, a document is invalid if it is not notarized. The notary doesn't really add value in a market sense, we have just decided that you need to go swear in front of another person that you are telling the truth or that the document is genuine. Is it really that much of a stretch to imagine that there will be a human review requirement for lots of things in the future, even if completely irrelevant (or actively detrimental) to the actual work?
Both of you are right. Individuals who manage to stay employed in the face of automation will be paid significantly more than was possible prior to automation, for the reasons the comment you're replying to pointed out. But labor as a whole will have less value. To give a simple example: if you own a warehouse and you replace 6 warehouse workers making $80,000 a year with a fleet of robots and 1 technician earning $200,000 a year, automation has minted a very comfortable living for someone while dropping your labor costs significantly.
There is no reason to expect capital to stop trying to squeeze labor just because inference takes the role of labor. With so much competition in the AI space, and no labor laws to protect it, why wouldn't capital drive the value of inference to the absolute bare minimum? And if the value of inference collapses to the bare minimum, who in their right mind would decide to serve inference?
Human labor has value because it is limited. If we keep up our current pace of data center buildouts and breakthroughs in model efficiency, then artificial intellectual labor will become functionally unlimited. Unlimited resources do not tend to hold their value. In that case, there are two potential outcomes: deliberate limitations on the output of artificial intellectual labor, which would require all parties involved to decide to lock themselves into their current market shares; or financial collapse of the whole ecosystem. What happens after that collapse is anyone's guess, but you have to imagine that it's the lenders who financed the whole thing who will be deciding what happens next rather than the failed tech businessmen.
No, that is not what I meant. No replacement for the person who left the organization has been hired. It would be inaccurate to say that they were "replaced" by AI, insofar as we had no plans to get rid of them if they had not decided to leave.
I am not a writer but I do have to write a lot for work. The person who used to edit things for me has not been replaced, and I am expected to use AI. It is very good, but it's not all the way there. I am now left to decide whether to take the time to do it right or to go back to doing what I actually get paid to do. I wish my organization realized how important both sides of that decision tree are: we really ought to be careful about how readers perceive what we put out there, AND this isn't my job and we should have someone who's even better than me at this.
But this is an old trend with technological developments. Thanks to e-mail, most people handle much or all of their own correspondence. There used to be specialized staff for that. They were very good at it! But they were shown the door once they ceased to be strictly necessary, and now we all waste a bunch of time fiddling with Outlook instead of doing our actual work.
I generally agree, but don't discount the impact of platforms like Patreon and Substack. Obviously, many authors of the past had patrons, but this was generally based on status and connections - and that's to say nothing of people who can support a writing career based off inherited wealth. But there is little historical precedent for an avenue for creatives to support themselves (or at least meaningfully supplement their income, potentially leading to less non-creative work hours and thus more time to hone the craft) using a decentralized base of support.
The trouble with this is that nobody else was making predictions about AI pre-transformers. Not many are making predictions about AI even now. Forecasting is a preoccupation of the rationalist crowd, and very few people gave much thought to AI before transformers. So the fact that they guessed right about certain things doesn't necessarily mean that the rest of their worldview is sound. A well-informed person who was inclined to making predictions about the future may have drawn similar conclusions without the sci-fi baggage.
I agree, and it's a particular shame in this instance, because what is startling about the HF incident - to me, anyway - isn't the degree of intelligence the agents exhibited but their persistence. I tend to believe that LLM architecture is not capable of producing a "superintelligence" in the way the LessWrong crowd defines that concept, but the combination of infinite stamina and infinite persistence is enough to cause some serious problems, even if intelligence plateaus right now.
FT frames this as some aggressive escalation tactic, but document retention letters are extremely standard practice. At this point they're basically a formality, as any former Apple employee at OpenAI really ought to know by now that they could get dragged into this. Hold letters can be aggressive if you send them before you've even filed a complaint, but if anything, Apple is late to the party with these.