Misconceptions about SB 1047

>If your model was trained on less than 10²⁶ flops and doesn't outperform those that were?

The speed at which the article glazes over one of the major issues with the bill is incredible.

The definition of "outperform those that were" (wording is pretty much the same in the bill) is part of the issue. The bill doesn't define any benchmarks, so whose to say tomorrow a new benchmark doesn't come out and your model trained at home on 3 4090s all of a sudden is the best at something.

And the bill also makes you responsible for what people do with your model. If it is decided that you didn't put in enough safeguards (again, no real definition) your liability goes through the roof.

Now companies are pretty likely to just ignore all this in the short term, but laws like this essentially make it just a matter of perspective to decide if someone is breaking the law or not. If you look hard enough from the right angle, any model could be argued to be in noncompliance.

It wastes taxpayer funds on enforcing a moat for Sam Altman, it establishes a fixed computational bound in a legal regulation, it tries to police a free speech activity because of possible harms (but not the harms directly), and it is likely to have negative national security implications as other (less regulated) regions deal with fewer lawyers as they advance the state of the art.

Wait, is this in favor of an AI regulation bill that doesn't do anything because the it would only cover models that don't exist? Is that right? Then what is the reason for this bill's existence?

My favorite provision for covered models is "make sure it can be shut down". Yes, perhaps this could be a concern for future "models", but circa 2024 LLMs do not have agency and are hosted on fairly traditional server farms with obvious power switches. This legislation may have referred to Hollywood films (Kubrick, Cameron, Wachowski et al) more than technological realities. While perhaps extreme shutdown measures might be desired to stop malicious software agents that depend upon inference from a covered model, there are other reasonable security measures available. The legislation seems to fundamentally misunderstand what large language models actually are.

The most dangerous thing about SB 1047 is that it is legislation without purpose. As the article rightly points out, it covers no real activities, and its requirement to be able to shut the machine down is, as far as anyone can figure, derived from watching The Terminator. A bill like this delegitimizes government altogether. I liken it to Dennis Kucinich's quixotic attempt to ban space-based mind control weapons.

The very obvious purpose of this bill is to provide a regulatory moat around the largest AI players, make no mistake. Didn’t openAI donate a large sum to this guy’s campaign pretty recently? Why aren’t they up in arms about this? Or any other big AI player? Doesn’t pass the smell test, especially given the barely veiled corrupt nature of the california political machine

> Covered models are those trained on more than 10²⁶ flops (a measure of computing power that, at current prices, is estimated to cost between tens and hundreds of millions of dollars), or projected to have similar performance on benchmarks used to evaluate state of the art foundation models. If your model was trained on less than 10²⁶ flops and doesn't outperform those that were? It is not a covered model.

I think the number needs to be indexed to some measure of readily available consumer hardware.

Otherwise, there are lots of computing capabilities that were estimated to cost 10s or millions of dollars (for example, the first hard drives were tens of thousands of dollars for a few megabytes and a terabyte of storage would have seemed like an astronomical amount projected at 10s of millions of dollars. Also for a long time, a teraflop was considered to be an astronomical amount of computing power which a high end desktop or laptop can now easily hit).

Not familiar with this magazine but the article is pure propaganda in favor of an absurd government bill. Feels odd to read. I hope people think about what would motivate one to write and publish a piece like this.

whats a concrete example about something I do with an AI model that I wouldn't be liable for today, but I would be after SB 1047 passes?

""" This will not apply to a small developer for years. At the point that it does, yes, if you make a GPT-5-level model from scratch, I think you can owe us some reports """

This is an insane way to legislate, so at best, it makes me think of underlying issues like corruption and incompetency.

---

More holistically, it's alarming from a view of what they chose to focus on vs not

I'm pro-regulation, but at the model provider tier, I rather see things like the equivalent of Net Neutrality and rules on platform providers not unfairly competing with app providers. That's the main new thing to be figured out, afaict, at the generic platform/utility provider level.

AI risk is real. At the platform level, the issue isn't models capable of doing bad things, but whether you are a platform for those doing it and how liable you are. Something like KYC is an everyone problem and exists, so I'm unclear why the new legislative land grab, that's a distraction.

I rather see targeted risk/harm-based legislation for app providers... Except that also already largely exists. The bank making loan decisions is regulated. Phishing is already illegal, irrespective of it using AI. Which model provider helps AI users stay compliant is a market thing. If the gov wants to be helpful, make clear standards at that application level with real consumer/business harm, and providers can decide if they help streamline those audits.

AI Platform providers should be liable for knowingly enabling customers to do bad things (or intentionally ignoring), but the nature of compliance changes when you take this stance... KYC again, which already exists. So again, corruption, incompetency, or what?

These politicians and lobbyists could have focused on regulating specific risks for specific industries, or ensuring liability and fair competition for model providers. But they took a very different road that feels more about anti-oss, anticompetitive, personal power, posturing, and weak risk reduction.

Nobody I know is furious with AI regulation. Indeed, we welcome it.