If you flip the switch then the trolley still proceeds and there’s still a 50% chance every human on the planet gets run over. You’re just not the one at the wheel.
Democratic societies have expressed a will for people to have inviolable rights, such that you may not appeal at will to the 'greater good/consequences' to harm others. It is a rejection of consequentialism.
Anthropic is in error for endorsing this logic. Every big trial reaffirms it since Nuremberg, you are responsible for the act you commit and your intent, and not what would or would not have happened otherwise.
Only under authority these ai companies do not have, would someone seriously consider harming the innocent as a lesser evil.
If you work for an AI company and you can't work safely, you must stop working.
I don't think "just make it illegal" is going to save us, that doesn't make me feel safe anyway. They may try that first because it's easy - create a regulatory body, sign some legislation, problem solved! [george-bush-mission-accomplished.jpg] But at this point I feel like some kind of Battlestar Galactica scenario is most likely - hopefully not quite as existential - but it will take a collective reaction to a traumatic event (a la Hiroshima/Nagasaki). Technical rather than (or in addition to) legal measures will be taken, like network partitioning and hardening. This is everyone's problem whether we like it or not.
(I'm not saying this "fatalism" should be used as an excuse by anyone working for any of these companies, it should give them pause that any bloodshed would still be squarely on their hands, but as an observer, people are gonna keep pushing until shit hits the fan. [jeff-goldblum-jurassic-park.gif] It's also not really about whether it's "appealing" or not, it's just trying to predict and anticipate different likelihoods...)
But my response is that we should still try even if we haven’t successfully stopped or slowed the development of technology in the past (which I don’t actually think is possible to know, I am personally aware of biological research to use brain cells as computers that was stopped by government intervention. But by definition we won’t know about most things that don’t exist).
Your narrow point is well stated that there is a categorical difference when technology is on the precipice of being created. And maybe AI is inevitable because it’s on just such a precipice. But I’d point out that AI is currently in a scaling cycle which provides an opportunity to slow the scaling and ensure scaling isn’t monopolistic thereby slowing the risk.
And I don’t think the evidence suggests that open source will continue to progress as rapidly absent distillation. I think if open ai and anthropic stopped releasing models broader progress would slow. But again your inevitable argument is persuasive
I agree that “just make it illegal” won’t work but shouldn’t then justify “so there’s nothing to do”.
There is lots we could do. For example we could make users and executives liable for their agents actions. We could assign a session ID to every agent tool call chain and if there is harm trace it to the user, we wouldn’t even need to read the conversation or contents of the tool calls or prompts it could be a simple rule, your ai agent causes personal liability. Users would be more careful or perhaps not even use AI in many cases that could inadvertently expose them to liability. We could extend liability to executives and employees of the Labs. They will be much more careful and more focused on alignment if they could be personally liable for the actions of their users and agents.
That is how the BigAI leads the society to the idea of necessity to relax the anti-monopoly laws when it comes to the Big AI - the main goal of all that "AI will kill you all" hysteria.
The examples for the dangers of AI is always comparisons with tools designed to destroy (which is understandable). But the problem with AI is it has the potential to create wealth for investors.
Thus those who have the opportunity to change course also have a conflict of interest in making that decision.
So the choice is: negotiate an unverifiable treaty (I.e. there's no way to verify compliance), or keep going as you are and try your best to not cause the destruction of humanity without slowing down.
The China argument is just a convenient scapegoat to convince the public that this isn’t just about greed.
For example, we don’t know if the pace of Chinese development of AI would have equal to what it is now if US companies weren’t racing against each other already.
I do fully believe that Chinese industry is lead by a desire to out pace the west. But I’m not convinced the same is true for the most American private entities. I think the reward model is different between businesses in America and businesses in China. I think the ambitions of CEOs is different. And I’m really not convinced that the CEOs of America are nearly as patriotic as they like to promote themselves to Trump and other political parties.
And that’s assuming the company even gets fined. More often than not, cases don’t even get that far.
Can you please explain what you mean by this because where I’m standing extremely dangerous companies are (and have been) running the economy
Exxon comes primarily to mind
Hofs bunny ranch is a famous brothel in NV
Booz Allen makes and maintains the nuclear fleet including the Sentinel ICBM
Textiles factories are globally known to be industrial slave camps for a non trivial portion of the supply. Even worse for Mica mines.
Etc…you can fill out the rest
Go open a brothel in NYC. It isn’t legal.
Go buy a nuclear bomb. Your ownership is illegal.
Go open textile mill in the abandoned buildings in North Carolina where children used to work and hire children to work the line. That will be illegal.
And no those aren’t exceptions, I could open a legal brothel in quebec, the netherlands, Brazil and be perfectly legal
If you think the US laws = ethics then I got nothing else to say
An exception by definition is a phenomenon with lower frequency than the general trend. Of the 50 us states only one has legal prostitution and only in highly regulated areas. Of the countries of the world it is similarly infrequent and thus an exception to the general rule that it is illegal.
Further the fact that it is illegal anywhere supports my point that it’s possible to regulate businesses based on their harm.
“Society doesn't allow dangerous companies to operate or exist”
That has been falsified thoroughly
Whether certain jurisdictions ban it or not is irrelevant to the original claim
Sure, that's why 99% of clothes are made offshore, but if we had something like tariffs on those (or requirements to prove that the actual factory adheres to labor and environmental standards), maybe more would be made "at home"? Similarly: Chinese cars undercutting US/German ones.
I mean there was just a video a couple of months ago of the giant room of sewing workers with cameras strapped to their faces capturing their hand movement so they could be automated
I saw entire cities emptied out over my lifetime.
I was in their offices at some point when that program was getting built out - Very much a Office Space bobs situation.
Philip Morris International? Monsanto? DuPont?
FTFY: Any self-respecting society with competent politicians.
The US has neither of that, and it shows everywhere you are looking.
> Go try to buy a tank and drive it into Manhattan. If we can prohibit that why can't we prohibit irresponsible AI development and deployment?
Because even if you had the tank, you can't make much money with it (unless you're a hitman, that is, but even for these, the payouts are measly). But if you are the surviving AI company in the usual VC playbook of "outcompete everyone else until society is completely and utterly hooked, then squeeze the customers by the balls"? The return on investment is virtually infinite. And that is what sustains the absurd valuations for all the AI companies.
Secondly, are you contending that progress in model efficiency and hardware just stops at whatever level you think is sufficient to prevent individuals or organizations from acquiring sufficient resources to run advanced models?
My point is that I would rather have 1000 labs training and serving inference than 2 because that would distribute the wealth creation more broadly rather than allowing OpenAI and anthropic to capture all the value, it would drive more innovation as a broader set of experiments are pursued in parallel.
It's just that so far nobody cares about explicit checks because they cost resources or slow down the models.
..."government of the people, by the people, for the people, shall not perish from the earth." -Lincoln, Gettysburg Adress
Unfortunately for AI, it still is. People still get to decide things at the city, town, village level.
Further, even if you are right, so what. Is that a reason to just accept bad public policy? That’s like saying, anyone can learn how to make smallpox at home with a basic lab set up so we should just ignore any safety measures.
Not necessarily, but it should probably inform that public policy. I think the problem is no one knows what the public policy should be assuming that scenario is true. Even if you, somehow, regulate away massive GPU cluster training making such future training impossible, existing models are already here. Further already training smaller models for things like images, speech, and other specialties is cheaper than the bigger models.
I agree that we need some regulations like everything else, but it’s not clear to me what the right policy should be. I think the European ai act is a fine start, but it’s clearly not enough nor does it necessarily limits the training portion just the application portion. Not to mention that the requirements there can be summarized into something like “you have to be careful, and show evidence you tried to be careful”.
That sounds reasonable. If applied to OpenAI and their agents multiple times breaking out of bad secured sandboxes, it should be enough.
But limiting the training?
There really is china and they have a different approach I suppose. But it is possible to talk with them.
Does it? To me it seems reasonable for OpenAI to argue they did try to be careful evident by the sandbox, they just made a mistake. Almost every 0day is categorized by something like that. We haven’t had a long history of establishing a negligence charge to security bugs. Could you be sued because you didn’t demonstrate “carefulness” and used Linux which is not written in a memory safe language and has had multiple CVEs before? How complicated should the chain of an exploit be to demonstrate “carefulness” to the courts?
> training
OP was the one suggesting that training could be controlled because massive gpu clusters could be regulated the way a nuclear power plant could. If you assume training costs won’t drop, then it’s feasible I guess. However, unlike a nuclear reactor, the final training result isn’t a radio active material, but rather an ordinary file that anyone can load and use for inference.
We used to run Microsoft Word and other popular applications with 8 MB RAM and it worked fine.
I am working on reducing ram requirements to run models but the weights are already compressed to the edge of the Shannon entropy boundary and might resist further compression.
Computing always gets cheaper and faster over time. We can argue about the exact rate of improvement but the results are inevitable and uncontrollable.
'When disagreeing, reply to the argument instead of calling names. "That is idiotic; 1 + 1 is 2, not 3" can be shortened to "1 + 1 is 2, not 3." '
Also LLM's have something to do with smallpox as a unrestricted LLM will happily guide any wannabe terrorist in how to make them.
if anybody was looking for a good reason for datacenters in space.
How far back into the history of computing do people who keep repeating shit like that know about? God.
Look at the thing in your fucking hand. Now go back just 20 years and see how things were.
Just because models and GPUs will be more advanced in the future doesn’t mean we need to let OpenAI and anthropic establish monopolies on the backs of stolen training data give unfettered access to the internet, the terminal and people’s file system while also allowing them to have limited liability protection behind the corporate veil. That’s a choice.
I don't see any of such entity would solve that problem. The government and regulator are in OpenAI and Anthropic's pocket, and I don't trust them a single bit on coming up with regulations. The consumers don't care; they just need something smart and cheap. And the society doesn't work either: each person is too busy fighting for their own survival rather than changing the system.
There are clear anti-trust mechanisms to prevent market capture and the emergence of asymmetric power. Go back and see how much nashing of teeth Lina Khan triggered in SV when she started to enforce antitrust law and then compare it to what the Pinkerton agency was doing in the transition from the guilded age to the progressive era.
There is a vocal segment of SV that wants the return of the guilded age. Marc Andreessen as said that explicitly. Those of us in SV that value free markets and recognize that the progressive era actually saved markets from their natural tendency to self destruct when winners capture markets and destroy competition that provides the incentive to innovate and drives the price setting function for efficiency.
In other words, the aspiring founders look at Google, Facebook, Tesla, etc. and think "how can I become one of them", not "how can I be different from them"? If you can find me examples of successful startups that think fundamentally different from those big techs, I'm very happy to be wrong and be corrected here.
Exactly, again, look at what COMPUTERS THEMSELVES used to be in the 1960s/1970s.
What the "P" in the PC stood for and why it was such a big deal
> In 2006, the mobile phone market was dominated by stylish flip phones, early music players, and physical keypads just one year before the iPhone changed the industry
It only takes me excavating massive amounts of uranium ore, building huge facilites packed with thousands of centrifuges that span multiple square miles, and paying all that infrastructure and workforce.
Your proverbial genie can be out of the bottle all you want, but it doesn't work without getting kicked in the ass by a very large golden boot.
And everyone had a fairly good idea what fission and fusion bombs would do once built. (Teller was worried Trinity might set off a nitrogen fusion reaction and kill all life on Earth, but Bethe and others proved him wrong before testing.)
No one knows what the limits of AI are. It's not just untested, it's unmodelled, and unplanned - build it first, worry about consequences later.
As Bruce Schneier recently discussed, law and tax law are code, just like source code. LLMs are great at finding holes in them. Illicit organizations looking to launder funds are most certainly interested in what AI can do for them.
Then we could require comprehensive logging of every tool call, model trace, chain of reasoning, and even tensor propagation all of which would be spot inspected like the CFTC does with commodity trading and settlement. We could have embedded auditors with specific risk analysis metrics like large banks do. We could limit tool calls to dedicated sandbox’s with a blanket prohibition on AI accessing user space. We could create a parallel internet for agents so they are only able to access Secure Enclave. Even if these measures aren’t 100% perfect they would reduce the risk.
We are very capable of putting good things in a box. We are just incapable of putting profitable things in a box.
Nobody (weirdly) proposes to forget about nuclear weapons, doesn't mean everybody should have one.
When you dream about flying a dragon to work, reality poses e.g. parking issues and insurance mismatch as obstructions. Maybe settle for a bike instead?
1. Trolleys actually don't usually have steering wheels.
2. People who actually hit trolley switches are not usually the ones at the driver's seat.