Hacker News Reader

Comment by pj_mukh | Hacker News Reader

Has anyone heard about any effect Meta has said would happen if Californias SB 1047 passes[1]?

Looking forward to continued updates and releases of Llama (and SAM!) from Meta.

[1] https://www.theverge.com/2024/8/28/24229068/california-sb-10...

I haven’t heard anything specific from Meta themselves, but I think the bill is short enough that we can reason as non-lawyers about it. Almost certainly they would have to stop releasing LLMs weights based on the very specific qualifications in the legislation. I don’t actually know what the specific size limit would be, but based on the translated $ value in the text of the bill it probably would cover their 70B+ models.

Disturbing approach to mitigating AI harms imo, this bill basically hopes it can limit the number of operators of an arbitrary model type so as to allow easier governance of AI model use. This ignores the reality that we already have large models openly released and easily modifiable (outside CA jurisdiction) which likely are capable of perpetuating “critical harms”, or that the information requirements to achieve the defined “critical harms” could be realized by an individual by simply reading a few books. There’s also no reason to simply assume that future models will require millions of dollars of compute to create; fulfilling the goals of this regulatory philosophy long term almost certainly requires the banning of general purpose compute to come close to the desired outcome of a supposed reduction in probability of some “critical harm” being perpetrated. We should be focusing on hardening society to the realities of the “critical harms” identified by this bill, rather than implicitly assuming the only reason we don’t see them as much irl is because everyone is stupid. The current paranoia wave around LLMs is just a symptom of people waking up to the fragility of the world we live in.

This is a good one for everything related to SB1047 https://pca.st/episode/44b41e28-5772-41c4-bcd7-5d7aa48d5120

Yoshua Bengio is a very respected scientist with well deserved reputation, but this discussion is upsetting… “academia now trains much smaller models… 10^26 FLOPs is 10 to the 26 floating point operations per second.. yes.. how big is that compared to GPT-4? It is much bigger than all the existing ones…” (flops has a different meaning: there is no per second in the law; one single H100 from last year performs 1e15 FLOPs per second; llama3.1 was close to the 1e26 limit this year, and the total training FLOPS of other models are not published; research could change once compute is even cheaper but state laws move at glacial speeds…).

It is disheartening so see the damage capacity in the hands of a couple of paranoic people who perhaps read the wrong scifi and had lots of power to influence others. If California passes this law, in a few years the world economy will be very different.

I think companies will simply quit doing business in California. They are killing the golden goose with all these regulations, just like they poison their own real estate markets allowing NIMBYs to dictate regulations that keep housing prices high and allow petty criminals to run rampant while cities like San Francisco continue to diminish and suffer

> I think companies will simply quit doing business in California

Sure they will. Just like how the auto manufacturers stopped trying to sell in CA after stricter emissions regulations and just like how companies stopped trying to serve European visitors after GDPR came into force. California is a $4T GDP, placing it just behind Japan and ahead of India in global rankings. I am sure they are happy to tell non-complaint companies to piss off and not let the door hit them on the way out. This is the canary in the coal mine for cowboy AI companies, once EU regulators get into the game their days of playing fast and loose with this tech will be over.

Unfortunately, it increasingly feels like the cowboy is the lobbyist in this picture and the victims of their scam are the regulators and people with semi religious beliefs. The specific limits in the law are possible to reach with this year’s hardware and enough money to buy a large gpu cluster and let it train for months. In five years the current limit in the law might be accessible by academia using government clusters and in ten years it may be accessible to patient home users, especially if some of the specialized hardwares that are now in the works find ways to commercialize and have mass appeal. Limiting the amount of compute by law is such an odd concept when we are in a phase of exponential growth.

The law isn’t permanent, if you notice that it doesn’t make sense in 5 years, you can still change it.

Dont underestimate the glacial kinetics of existing laws. I dont recall many examples of legal regulation that took lots of lobbying and years of planning to pass, to then be changed within years. Sometimes non-permanent laws, such as executive orders, change when executives change, but laws are typically only created and stay until they are either deemed unconstitutional or completely out of sync with reality, and even then only if there is nothing more important or new to write laws about. The more common end of bad laws is for executives to decide to not enforce them, eg outdated federal or state laws about drugs, but it takes a lot of momentum and time to undo a bad law, much more time than to write it in the first place. Just like with bad code in large codebases, new code is easier to write and brings more glory than carefully eliminating or amending poor code.

IMHO it's not, it just parrots the same old arguments for "safety", arguing against straw-men and framing the other side as having wrong assumptions about AI safety/being unfair/etc, all while not going into the principled counter-arguments and their own assumptions at all.

Here are some counter-points:

Regulation:

- Very little effort is made to evaluate risk of over-regulating, regulatory capture and counterproductive wrong regulation

- The downside of under-regulating is vastly overemphasized, most arguments boil down to "we have to act FAST now or x BAD thing might happen"

- The risk of over/wrongly regulating is vastly under emphasized with the same FUD reasons.

- according to one of the many straw-men argument in the pod I'm a libertarian against any and all regulation because I criticize possible regulatory capture, I would enthusiastically support regulation that foundation models have to be:

-- given freely to public researchers/academics for in-depth independent safety-research

-- open weighted after a while (e.g. after ~ a year, safety concerns should be mostly ruled out and new generations are out so ROI is already likely there. [e.g. there's NO safety reason at all for ClosedAI to not release gpt-3.5, llama3 is better already])

Proposed FLOP cut-off of SB 1047:

- according to the pod, the cut off is much more advanced than anything currently released.

- The 10^26 FLOP cutoff is way to low, llama-405b is ~4×10^25 FLOPs

- 405B is maybe 20% smarter than 70B, while taking over an order of magnitude more FLOPS to train, the cutoff itself is very likely not much smarter than the current SOTA.

- IMO none of the current SOTA models are very dangerous, but kill switch regulation is.

Kill-Switches:

- SB 1047 is (non-explicitly) calling for kill-switches over the cut-off due to liability of the model creators and market dynamics

- Any kill-switch regulation means a complete dead-end to any advanced open-weights AI. This means that huge corporations and governments will control any and all advanced AI-development. This is top-down control of the maths you are allowed to run on your computer IMO that is Orwellian as fuck.

China:

- mentioning china is FUD 101, it's basically AI's "think of the children"

- If they think they can stop china from building their own advanced LLMs, they're delusional. This regulation might even help them to get there faster. They don't even need to steal, there maybe a year or two behind the SOTA and catching up fast.

I just don't get how so many people on a site with "hacker" in the name want to make it impossible to hack on these things for anyone not employed by the big corporate AI research labs.

Reply on news.ycombinator.com