Call-to-Action on SB 1047 – Frontier Artificial Intelligence Models Act
affuture.org
affuture.org
It's too late to stop "deep fakes". That technology is already in Photoshop and even built into some cameras. Also, regulate that and Hollywood special effects shops may have to move out of state.
As for LLMs making it easier to people to build destructive devices, Google can provide info about that. Or just read some "prepper" books and magazines. That ship sailed long ago.
Real threats are mostly about how much decision power companies delegate to AIs. Systems terminating accounts with no appeal are already a serious problem. An EU-type requirement for appeals, a requirement for warning notices, and the right to take such disputes to court would help there. It's not the technology.
Right, there is no issue with how "smart" ML models will get or whatever ignorant framing about intelligence and existential risk gets made up by people who don't understand the technology.
The real concern is dumb use of algorithmic decision making without recourse which is just as valid whether it's an if statement or a trillion parameter LLM.
However, those can and are tracked. The thing making them nervous is the ability to do that on your own with no possible way for someone to track or catch you. Same with deepfakes. They don't care if you are doing it with photoshop, because that can be reviewed. They care that you can do it and not be caught/stopped/punished for it.
>SB 1047 creates an unaccountable Frontier Model Division that will be staffed by EAs with police powers, and which can throw model developers in jail for the thoughtcrime of doing AI research. It’s being fast-tracked through the state Senate. Since many cloud and AI companies are headquartered in California, this will have worldwide impact.
Of course that is scare propaganda, but when you put it with what the Federal govt is doing here[0], it makes it pretty clear that the real worry is people have access to "dangerous" information with no oversight. I can imagine policing agencies at every level getting very nervous with lone-wolf or tiny militia types getting access to information without any triggers flipping and alerting them and with no way to get any evidence if they do want to arrest them for something.
[0]https://www.msn.com/en-us/news/us/us-homeland-security-names...
The danger of ai has nothing to do with what the average Joe might try to do, it has everything to do with what soulless corporations are doing to you right now and how it enables them to be even worse in the future.
Right now your roof is being scanned by aircraft with cameras and AI is being used to determine how old it is and if there are tree branches nearby. They're also looking at and classifying objects in your back yard to determine safety risks. It's not horribly accurate but because of the scale it doesn't matter to the companies, you just get fucked. Accidentally bag something you didn't scan at the self checkout? They have AI for that too, there are multiple reports of people being hunted down and charged with theft for simple mistakes.
Your chances of having your life ruined of degraded because of AI are massively higher than your chances of being hurt by a random individual using it to build destructive devices.
[0]https://www.msn.com/en-us/news/us/us-homeland-security-names...
Why not both? With the story of a high school principal being framed by a coworker who deep faked a racist anti-semitic rant that the principal didn't say, I'd say the danger of AI also has to do with what an average Joe that wants to cause you harm can do. That doesn't diminish the threat from corporations, but a jilted lover can now ruin your life in additional ways.
https://www.washingtonpost.com/dc-md-va/2024/04/26/baltimore...
Yes there are dangers there but they ultimately come down to evidentiary standards. We can't do the thing we always do where all risk is perceived based off of extremely rare incidents so we destroy everyone's privacy while the stuff actually harming people at scale is ignored.
https://www.answer.ai/posts/2024-04-29-sb1047.html
The EFF have also prepared a submission:
https://www.context.fund/policy/2024-03-26SB1047EFFSIA.pdf
A key issue with the bill is that it criminalises creating a model that someone else uses to cause harm. But of course, it's impossible to control what someone else does with your model -- regardless of how you train it, it can be fine-tuned, prompted, etc by users for their own purposes. Even then, you can't really know why a model is doing something -- for instance, AI security researchers Arvind Narayanan and Sayash Kapoor point out:
> Consider the concern that LLMs can help hackers generate and send phishing emails to a large number of potential victims. It’s true — in our own small-scale tests, we’ve found that LLMs can generate persuasive phishing emails tailored to a particular individual based on publicly available information about them. But here’s the problem: phishing emails are just regular emails! There is nothing intrinsically malicious about them. A phishing email might tell the recipient that there is an urgent deadline for a project they are working on, and that they need to click on a link or open an attachment to complete some action. What is malicious is the content of the webpage or the attachment. But the model that’s being asked to generate the phishing email is not given access to the content that is potentially malicious. So the only way to make a model refuse to generate phishing emails is to make it refuse to generate emails.
Nearly a year ago I warned that that bills of this kind could hurt, rather than help safety, and could actually tear down the foundations of the Enlightenment:
Build a model that is trained on the corpus of gun designs.
Should be an interesting court case and social experiment.
The reasonable take of course is that the tools are never to blame, they are just tools after all. Blame the bastard using the tools for nefarious ends, whether it's guns or "AI" or whatever else the case may be.
The Oklahoma City bombing 1995
Most people with a high school level of chemistry and a trip to the library can cause a lot of damage.
David Hann https://en.wikipedia.org/wiki/David_Hahn the radioactive Boy Scout single handedly created a superfund site.
This will quickly turn into a first amendment case and die in court I would think.
I am against laws and systems of government that turn me into a suspect just for learning information. It is not ok that a secret investigation into our private lives is triggered simply by being curious.
I'm not under the impression that the EA movement is better suited to steward AI development than other groups, but even assuming they were, there is no chance for an initiative like this to work unless every country agreed to it and followed it.
California has too much regulatory burden and taxation.
Am I jumping to conclusions and is there a different interpretation you think I should be coming away with?
But there is also another interpretation, which is that the new thing is going to happen in whatever place has the least stringent rules anyway, so more stringent rules don't improve safety, they just deprive your jurisdiction of any potential rewards from keeping the activity local, and provide people in other jurisdictions the benefit of the influx of people you're inducing to leave.
I'm not sure in the general context your first came is super applicable, given that there's a lot of exposure and worry about this topic. Laws and regulation can apply to use as well as development, and large markets can have outsized effects when they require things (such as how CA emissions laws and GDPR have), and I'm not sure worrying about chasing away business is a worthwhile concern when many people are very afraid of a societal consequences of the thing in question.
For what it's worth I don't really follow the same stance when it comes to military technology, because that's meant to be used in a situation when local (which in that case can be national) laws have little or no sway, but I'm open to arguments about how viewing them differently isn't useful.
It doesn’t make any sense to talk about the number of regulations. What matters is what those regulations are. Likewise, it doesn’t make sense to talk about the amount of taxation without talking about who is being taxed.
At a glance it looks like it's not going to affect AI projects that are basically consumers of existing models, which is most projects.
Could you at least consider that the group’s entire premise seems like nothing more than the post hoc rationalization of a bunch of wealthy educated elites with low social and emotional intelligence? The levels of tone deaf I perceive as someone who doesn’t have, can’t have that much wealth to even begin my journey in their little club are enormous.
There’s very little that is subtle about it and it’s frankly offensive and _clearly_ used as a justification for insecure Bay Area “liberals” who find themselves with lots of money and a political identity that makes them insecure about that fact.
The answer? You’re actually saving mankind with your money! It’s just simple Bayesian logic! The same thing that got you here! (Spoiler alert: it was more to do with luck than skill).
If you’re on board, I guess it’s not as offensive? But for me and others, it’s like elites trying to brag about how great they are while the rest of us fight for scraps.
So when you see someone with an axe to grind, maybe consider that EA’s messaging is not as universally appealing as you think, and may even be outright tone deaf enough to cause one to reasonably find it disgusting.
> and he was reportedly ousted from the board by EA-aligned folk.
It seems naive to me to assume that members of EA in positions of power don’t secretly have their own motivations. Furthermore it’s not very “Bayesian” to assume that a implies b here with so many hidden variables at play.
I'm not sure if this was directed at my comment, but I'll clarify that my comment wasn't in defense of EA. My position is: setting up a regulatory moat is not a strategy exclusive to EA. Many incumbents - including sama - are overtly (and likely coverly) attempting regulatory capture, regardless of their 'politics'
> SB 1047 creates an unaccountable Frontier Model Division that will be staffed by EAs with police powers, and which can throw model developers in jail for the thoughtcrime of doing AI research
If the bill says nothing about who will be staffing this agency, and there are indeed no ties to EA (which seems unlikely to me if EA is behind the bill), then the author of the article is doing us a disservice by misrepresenting it.
If it affects the base projects (especially the open source ones like Llama) then it affects the consumers. And it certainly looks like it's planning to affect the base projects, in a lot of negative ways.
If this bill passed in any way remotely similar to what it is now, Meta would have to entirely stop releasing open source Llama updates.
Which is perhaps the intent of the legislation.
Also, from a moral philosophy point of view, "this is wrong but if we don't do it, someone else will collect the profits and the bad thing will happen anyway - hence we'll do this wrong thing" is not sound. Applied generally it would lead to a lot of bad stuff (moral race to the bottom).
Whether or not you believe this is the case with AI is not what I'm getting at, but a lot of people believe AI has the potential to cause more damage than nuclear weaponry technology, as well as the potential to suspend aging, end all natural causes of death, regrow limbs, fix any bodily ailments.
There will be countries that embrace AI out of potential for the latter, and capitalists who do their capitalist thing and make as much money off of the perceived latent power.
There is no prospect of achieving a truly international consensus (meaning applied the same way by every country, we don't even have that with copyright, though that's the closest example I can think of) on AI policy. Given that, do you really think the U.S. is going to sit by and watch as the majority of AI research is conducted in other countries?
From this document, they define:
“Artificial intelligence model” means an engineered or machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs that can influence physical or virtual environments and that may operate with varying levels of autonomy.
My point was that it's a spectrum, and the law doesn't seem to give guidance on where to draw the line on that spectrum. The hand mixer was just a clearly absurd example on the far opposite end of it, to show its breadth.
So back to your question; some improvements in my mind might be:
(1) Don't phrase this as an AI topic at all. Make laws about the safety of automated systems of all kinds which have health & safety implications — we already have lots of laws for cars (whether AI driven or not), medical equipment, and yes, even kitchen appliances (: Then, the definition of "AI Model" is irrelevant.
(2) If we do want something specific to AI, then the definition should be more specific. The definition could involve it being a stochastic process (unlike much other software), having inner workings that are poorly understood even by experts (unlike much other software), and whose logic is developed statistically from training data rather than being hand-designed (unlike much other software (or kitchen appliances!)).
If the user stops the process when they want (instead of the process self regulating), it's hard to argue the process is doing anything in isolation.
It's not that big
(1) The artificial intelligence model was trained using a quantity of computing power greater than 10^26 integer or floating-point operations.
(2) The artificial intelligence model was trained using a quantity of computing power sufficiently large that it could reasonably be expected to have similar or greater performance as an artificial intelligence model trained using a quantity of computing power greater than 10^26 integer or floating-point operations in 2024 as assessed using benchmarks commonly used to quantify the general performance of state-of-the-art foundation models.
…and have the following:
“Hazardous capability” means the capability of a covered model to be used to enable any of the following harms in a way that would be significantly more difficult to cause without access to a covered model:
(A) The creation or use of a chemical, biological, radiological, or nuclear weapon in a manner that results in mass casualties.
(B) At least five hundred million dollars ($500,000,000) of damage through cyberattacks on critical infrastructure via a single incident or multiple related incidents.
(C) At least five hundred million dollars ($500,000,000) of damage by an artificial intelligence model that autonomously engages in conduct that would violate the Penal Code if undertaken by a human.
(D) Other threats to public safety and security that are of comparable severity to the harms described in paragraphs (A) to (C), inclusive.
…In which case the organization creating the model must apply for one of these:
“Limited duty exemption” means an exemption, pursuant to subdivision (a) or (c) of Section 22603, with respect to a covered model that is not a derivative model that a developer can reasonably exclude the possibility that a covered model has a hazardous capability or may come close to possessing a hazardous capability when accounting for a reasonable margin for safety and the possibility of posttraining modifications.
A harms-based approach, regardless of the model used, seems more able to be put into practice.
[1] https://www.whitehouse.gov/briefing-room/presidential-action...
So in practice, only the flops criteria matters. Which means only giant companies with well-funded legal departments, or large states, can build these models, increasing centralization and control, and making full model access a scarce resource worth fighting over.
I've been actually thinking there should be a bounty for a real hazardous use of AI identified. The problem would be defining hazardous (which would hopefully itself spur conversation). On one end I imagine trivial "hazards" like what we test models with today (like asking to build a bomb) and on the other it's easy to see there could be a shifting goalposts thing where we keep finding reasons something that technically meets the hazard criteria isn't reall hazardous.
There is freedom of speech regardless if it's written in English or C.
It was also tried in a very different time. Given that we can't even allow free speech on digital platforms today, I'm not sure that many courts would allow for free speech claims to fall under the first amendment.
- Developers must assess whether their AI models have hazardous capabilities before training them. They must also be capable of promptly shutting down the model if safety concerns arise.
- Developers must annually certify compliance with safety requirements. They must report any AI safety incidents to a newly created Frontier Model Division within the Department of Technology.
- Cluster Operation Regulation: OOpolicies to assess whether customers intend to use the cluster for deploying AI models. Violations may lead to civil penalties.
- A new division within the Department of Technology will review developer certifications, release summarized findings, and may assess related fees.
- The Department of Technology will establish a public cloud computing cluster named CalCompute, focusing on safe and secure deployment of large-scale AI models and promoting equitable innovation.
My outsider's understanding is that we really don't know specifically how the models learn what they learn or why they give specific answers. Is it possible that we could even know whether a model could present hazardous capabilities prior to training it? Or after it for that matter?
Alliance for the Future is a lobby group of effective accelerationists who endorse some of Marc Andreesen and Peter Thiel's views in their manifesto, and based on that plus this article, they seem to oppose the bill entirely.
A place to start for a breakdown of what's in the bill is the Context Fund analysis that AFTF links to. That analysis cites similar critiques from EFF, the Software & Information Industry Association, and others. All of these are from the perspective of voting against or substantially changing the bill.
I haven't found "pro bill" opinions as easily, but I haven't been plugged into the conversations around this, so I'm missing anything that doesn't appear on the first few pages of Google or DDG.
> (b) “Artificial intelligence model” means an engineered or machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs that can influence physical or virtual environments and that may operate with varying levels of autonomy.
So if I hand-write instructions to make a chemical weapon, and aggressively "fine-tune" Llama 7B to output those instructions verbatim regardless of input, Meta is liable for releasing a model with hazardous capabilities?
The line about autonomous actions is only item C in the list of possible harms. It is separate from item A which covers chemical weapons and other similar acts.
The point is that, because fine-tuning can trivially induce behavior that satisfies the standard for "hazardous capability" in any model, the law effectively makes it illegal to release any covered model.
It seems dumb to have a separate classification for harms caused by trained AI models. The training aspect doesnt seem to limit liability at all. A judge might rule differently, but thats why the justice system is built such as it is, to make intelligent decisions based on the specific facts of a case.
I am betting that software that causes some significant harm is already outlawed. So this whole thing is just a waste of time.
It's incredibly difficult to imagine doing a good job of regulating model training, especially in a few years when the available flops are high enough that this limit is being hit often.
It's much more straightforward to regulate actions: constructing WMD is illegal, synthesizing drugs is illegal, etc.
If the state wants to tighten up its laws about various activities, go for it. That's the right place to act. Injecting itself into the model training process seems very unlikely to yield any substantive benefits and very likely to hinder progress.
Interesting, we don't have transparent, uniform, publicly available price schedule for healthcare and other basic needs (electricity, e.g. see PGE).
Something is fishy here.
If you're actually worried about AI we need to ban any generative AI that can replicate a specific person's voice or appearance. Beyond that I don't see any immediate danger.
How long has Musk been promising full self driving in the next X months (while making people pay $10k for it)? Anyone taking his word as anything close to reality is a fool
Well Sam literally asked for 7 trillion dollars to build out the semiconductor industry enough to support AI. That implies AI is worth that kind of investment. He's a smart guy, but I saw the video and IMHO he really sounded foolish in that moment.
Let the hype continue...
Florida is growing too fast as it is.
I don't know if all the language in this bill does what we need, but I'm against letting large corporations like a META or X live test whatever they want on their end users.
Calling out derivative models are exempt sounds good; only new training sets have to be subjected to this. I think there should be an academic limited duty exemption, models that can't be commercialized likely don't need the rigor of this law.
I guess I don't agree with affuture.org and think we need legislation like this in place.
From the post. > We need your help to stop this now.
I do not want to stop this bill; I want it revised. If that is not understood, then I suspect my primary goal for safer, less biased models is at odds with your primary goal of unregulated innovation.
At least if we continue to discuss this as a binary where agreeing with this affuture.org post means killing this legislation (as the post asks for) and not replacing it.
And I'm not sure if it's even possible to do something drastic enough at this point - regulating datacenters would just make companies move to other countries, just like this would probably just make companies move out of CA.
Why do you think that?
> regulating datacenters would just make companies move to other countries
To say nothing of the potential issues regarding free society going down this route will yield - and has arguably already yielded.
I agree with your point and most other points about the negatives of regulating compute but like, if the other side of the scale is species-level genocide, does any of it matter?
Possibly not, but I have to disagree with your assessment of our future. It doesn't seem as clear to me that we couldn't co-exist with an AI superintelligence (putting aside any arguments about what that even means for now)
Bad lawmakers commit this fallacy all the time.
Write your good law idea down and send it to a lawmaker who will act.
By the way, this is how the EU does things and that's why we're always behind on anything tech :)
For example, the EU regs for flight delays are a great example of consumer protection that is actually beneficial. You get paid cash compensation (which often exceeds the face value of the ticket) if you’re delayed more than a certain amount.
Seriously, the death and destruction caused by lead, morphine and mercury in everyday things was not a joke.
What do you mean? You can download llama.cpp or Stable Diffusion and run it on your ordinary PC right now. People make variants using LoRA adapters and things with relatively modest resources. Even creating small specialized models from scratch is not impossibly expensive and they often outperform larger generalized models in the domain they're specialized for.
Creating a large model like llama or grok takes a lot of resources, but then it's entities with a lot of resources that create them. Both of those models have open weights.
For as long as you don't distribute the model and you only use it for yourself, you don't fall under this definition (if I understand correctly).
If you would distribute a mod of the model you would fall under the restrictions of this bill. Which is why I asked the original question: are people even doing this?