Hacker News Reader

Comment by worstspotgain | Hacker News Reader

Excellent move by Newsom. We have a very active legislature, but it's been extremely bandwagon-y in recent years. I support much of Wiener's agenda, particularly his housing policy, but this bill was way off the mark.

It was basically a torpedo against open models. Market leaders like OpenAI and Anthropic weren't really worried about it, or about open models in general. Its supporters were the also-rans like Musk [1] trying to empty out the bottom of the pack, as well as those who are against any AI they cannot control, such as antagonists of the West and wary copyright holders.

[1] https://techcrunch.com/2024/08/26/elon-musk-unexpectedly-off...

View on HN

> Excellent move by Newsom. [...] It was basically a torpedo against open models.

He vetoed it in part because the threshold it applies to at all are well-beyond any current models, and he wants something that will impose greater restrictions on more and much smaller/lower-training-compute models that this would have left alone entirely.

> Market leaders like OpenAI and Anthropic weren't really worried about it, or about open models in general.

OpenAI (along with Google and Meta) led the institutional opposition to the bill, Anthropic was a major advocate for it.

> He vetoed it in part because the threshold it applies to at all are well-beyond any current models, and he wants something that will impose greater restrictions on more and much smaller/lower-training-compute models that this would have left alone entirely.

Well, we'll see what passes again and when. By then there'll be more kittens out of the bag too.

> Anthropic was a major advocate for it.

I don't know about being a major advocate, the last I read was "cautious support" [1]. Perhaps Anthropic sees Llama as a bigger competitor of theirs than I do, but it could also just be PR.

[1] https://thejournal.com/articles/2024/08/26/anthropic-offers-...

> I don't know about being a major advocate, the last I read was "cautious support" [1]. Perhaps Anthropic sees Llama as a bigger competitor of theirs than I do, but it could also just be PR.

This seems a curious dichotomy. Can we at least consider the possibility that they mean the words they say or is that off the table?

Just two spitballing conjectures, not meant to be a dichotomy. If you have first-hand knowledge please contribute.

One of the very most apt descriptions. One company trying to raise alarms about another company in the same business. I have been following this since IBM released $67,000 "Intellect" "Business Intelligence." and Lotus HAL.

He's a politician, and his stated reason for the veto is not necessarily his real reason for the veto.

Makes perfect sense since his elected based on public positions

This is the ideal, but it's often false in meaningful ways. In several US elections, for example, we've seen audio leaked of politicians promising policies to their donors that would be embarrassing if widely publicly known by the electorate.

This suggests that politicians and donors sometimes collude to deliberately misrepresent their views to the public in order to secure election.

Sometimes? lol

worse.. a first-hand quote from inside a California Senate committee hearing chamber.. "Don't speak it if you can nod, and don't nod if you can wink" .. translated, that means that in a contentious situation with others in the room, if allies can signal without speaking the words out loud, that is better.. and if the signal can be hidden, better still.

This is an old saying in politics and you're misinterpreting it - it's not about signaling to allies, it's about avoiding being held to any particular positions.

You're also missing the first half, "don't write if you can speak, don't speak if you can nod, and don't nod if you can wink." The point is not to commit to anything if you don't have to.

Anthropic was championing a lot of FUD in the AI area

Newsom vetoed the bill as a nod to his donors plain and simple. Same reason he just signed a bill allowing a specific customers at a single venue to be served alcohol later than 2 AM. Same reason he carved out a minimum wage exemption for Panera. Same reason he signed a bill to carve out a junk fee exemption specifically for restaurants.

He's just planning for a post-governor career.

Same reason the governor appointed public utility commission has allowed PG&E to raise rates 4 times in a single year without legitimate oversight. Yea unfortunately all roads point to his donors with this smooth talker, cost of living be damned.

On the east coast we don’t need the government to control electricity prices. And our electricity is cheaper. Go figure.

Companies like Dominion Energy, Duke Energy, and Consolidated Edison are regulated by state utility commissions, same as in California.

Why did New Jersey get more toxic waste dumps than California has lawyers? They got first choice. We will trade Newsom for a failed reactor. Please.

Oh yeah I forgot this one — basically making it easier to force neighborhoods to abandon their natural gas infrastructure. Something I'd be in favor of were it not for the constant stream of electric rate hikes.

https://www.kqed.org/news/12006711/newsom-signs-bill-to-help...

When you're a politician and have a business hobby

>> He's just planning for a post-governor career.

After this year, many democrats are as well - which is why Harris had such a hard time finding a VP and took Walz who was like the last kid you pick on your dodgeball team.

The presidential race in 2028 for the Democrats is going to have one of the deepest benches for talent I've seen in a long time. Newsom and Shapiro will be at the top of the list for sure.

But I agree, Newsom has been making some decisions lately that seem to indicate he's trying to clean up his image and look more "moderate" for the coming election cycles.

> Newsom and Shapiro will be at the top of the list for sure.

Neither has genuine appeal. Shapiro is a really, really poor speaker and has few credentials except as a moderate. Newsom is the definition of coastal elite. Both have spoken, neither have been heard.

Oops, I had mixed up Shapiro with Gretchen Whitmer (another of these "rising stars"). Shapiro just does white Obama.

Wow. Now that was funny. Thanks.

2032, nice try though. Besides, we're not going to need to vote anymore otherwise, remember? The 2028 Democratic Primary would be a pro-forma affair between Barron vs. Mayor McCheese.

why would Google, Microsoft and OpenAI oppose a torpedo against open models? Aren't they positioned to benefit the most?

If there was just one quasi-monopoly it would have probably supported the bill. As it is, the market leaders have the competition from each other to worry about. Getting rid of open models wouldn't let them raise their prices much.

So if it's not them, who is the hidden commercial interest sponsoring an attack on open source models that cost >$100mm to train? Or does Wiener just genuinely hate megabudget open source? Or is it an accidental attack, aimed at something else? At what?

Like I said, supporters included wary copyright holders and the bottom-market also-rans like Musk. If your model is barely holding up against Llama, what's the point of staying in.

And two of the three godfathers of AI, and all of the AI notkillaboutism crowd.

Actually, wait, if Grok is losing to GPT, why would Musk care about Llama more than Altman? Llama hurts his competitor...

The market in my argument looks like OpenAI ~ Anthropic > Google >>> Meta (~ or maybe >) Musk/Alibaba. The top 3 aren't worried about the down-market stuff. You're free to disagree of course.

He can't compete with Anthropic and OpenAI because their models are much better. Llama is the competitor with the capability closest to Grok.

Claude, SSI, Grok, GPT, Llama, …

Should we crown one the king?

Or perhaps it is better to let them compete?

Perhaps advanced AI capability will motivate advanced AI safety capability?

What are the economic incentives for AI safety?

It's an interesting thought that as AI advances, and becomes more capable of human destruction, programmers, bots and politicians will work together to create safety for a large quantity of humans

"AI defense corps"

You think they want to compete? None of them want to compete. They want to be a protected monopoly.

The bill included language that required the creators of models to have various "safety" features that would severely restrict their development. It required audits and other regulatory hurdles to build the models at all.

If you spent $100MM+ on training.

Advanced technology will drop the cost of training.

The flop targets in that bill would be like saying “640KB of memory is all we will ever need” and outlawing anything more.

Imagine what other countries would have done to us if we allowed a monopoly like that on memory in 1980.

No, there are two thresholds and BOTH must be met.

One of those is $100MM in training costs.

The other is measured in FLOPs but is already larger than GPT-4, so the “think of the small guys!” argument doesn’t make much sense.

Tell that to me when we get to llama 15

What?

“But the big guys are struggling getting past 100KB, so ‘think of the small guys’ doesn’t make sense when the limit is 640KB.”

How do people on a computer technology forum ignore the 10,000x improvement in computers over 30 years due to advances in computer technology?

I could understand why politicians don’t get it.

I should think that computer systems companies would be up in arms over SB 1047 in the same way they would be if the government was thinking of putting a cap on hard drives bigger than 1 TB.

It puts a cap on flops. Isn’t the biggest company in the world in the business of selling flops?

It would be crazy if the bill had a built-in mechanism to regularly reassess both the cost and FLOP thresholds… which it does.

Inversely to your sarcastic “understanding” about politicians’ stupidity, I can’t understand how tech people seem incapable or unwilling to actually read the legislation they have such strong opinions about.

If your goal is to lift the limit, why put it in?

We periodically raise flop limits in export control law. The intention is still to limit China and Iran.

Would any computer industry accept a government mandated limit on perf?

Should NVIDIA accept a limit on flops?

Should Pure accept a limit on TBs?

Should Samsung accept a limit on HBM bandwidth?

Should Arista accept a limit on link bandwidth?

I don’t think that there is enough awareness that scaling laws tie intelligence to these HW metrics. Enforcing a cap on intelligence is the same thing as a cap on these metrics.

https://en.m.wikipedia.org/wiki/Neural_scaling_law

Has this legislation really thought through the implications of capping technology metrics, especially in a state where most of the GDP is driven by these metrics?

Clearly I’m biased because I am working on advancing these metrics. I’m doing it because I believe in the power of computing technology to improve the world (smartphones, self driving, automating data entry, biotech, scientific discovery, space, security, defense, etc, etc) as it has done historically. I also believe in the spirit of inventors and entrepreneurs to contribute and be rewarded for these advancements.

I would like to understand the biases of the supporters of this bill beyond a power grab by early movers.

Export control flop limits are designed to limit the access of technology to US allies.

I think it would be informative if the group of people trying to limit access of AI technology to themselves was brought into the light.

Who are they? Why do they think the people of the US and of CA should grant that power to them?

Wait sorry, are you under the impression that regulated entities get to “accept” which regulations society imposes on them? Big if true!

Your delusions and lack of nuance shown in this very thread are exactly why people want to regulate this field.

If developers of nuclear technology were making similar arguments, I bet they’d have attracted even more aggressive regulatory attention. Justifiably, too, since people who speak this way can't possibly be trusted to de-risk their own behavior effectively.

Moore's law is dead though.

Cost as a perf metric is meaningless and the history of computer benchmarks has repeatedly proven this point.

There is a reason why we report time (speedup) in spec instead of $$

The price you pay depends on who you are and who is giving it to you.

That’s why there are two thresholds.

Cost per FLOP continues to drop on an exponential trend (and what bit flops do we mean?). Leaving aside more effective training methodologies and how that muddies everything by allowing superior to GPT4 perf using less training flops, it also means one of the thresholds soon will not make sense.

With the other threshold, it creates a disincentive for models like llama-405B+, in effect enshrining an even wider gap between open and closed.

Why? Llama is not generated by some guy in a shed.

And even if it were, if said guy has such amount of compute, then it's time to use some of it to describe the model's safety profile.

If it makes sense for Meta to release models, it would have made sense even with the requirement. (After all the whole point of the proposed regulation is to get some better sense of those closed models.)

Also the bill was amended NOT to extend liability to derivative models that the training company doesn’t have effective control over.

Both thresholds have a system to be adjusted.

If the danger is coming from the amount of compute invested, then cost of compute is irrelevant.

A much better objection to static FLOP thresholds is that as data quality and algorithms increase, you can do a lot more with fewer FLOPs / parameters.

But let’s be clear about these objections - they are saying that FLOP thresholds are going to miss some harms, not that they are too strict.

The rest is arguing about exactly where the FLOP thresholds should be. (And of course these limits can be revised as we learn more.)

Or used a model someone open sourced after spending $100M+ on its training?

Like if I’m a startup reliant on open-source models I realize I don’t need liability and extra safety precautions but I didn’t hear any guarantees that this wouldn't turn off Meta from releasing their models to me if my business was in California?

I never heard any clarifications from the Pro groups about that

The bill was amended for training companies to have no liability for derivative models they don’t have control over.

There’s no new disincentive to open sourcing models produced by this bill, AFAICT.

All that means that the barriers for entry for startups skyrocket.

Startups that spend >$100mm on one training run...

There are startups and startups, the ones that you read on media are just a fraction of the worldwide reality.

You misread my intention, I think. I was pointing out that startups that don't raise enough to train a GPT-4 are excluded from the bill, that requires >$100mm expenses for a model to be covered.

Fair enough. Hope all is well there.

No offence taken, just a misunderstanding. I didn't downvote you.

Yeah, I think the argument that "this just hurts open models" makes no sense given the supporters/detractors of this bill.

The thing that large companies care the most about in the legal realm is certainty. They're obviously going to be a big target of lawsuits regardless, so they want to know that legislation is clear as to the ways they can act - their biggest fear is that you get a good "emotional sob story" in front of a court with a sympathetic jury. It sounded like this legislation was so vague that it would attract a hoard of lawyers looking for a way they can argue these big companies didn't take "reasonable" care.

Sob stories are definitely not covered by the text of the bill. The "critical harm" clause (ctrl-f this comment section for a full quote) is all about nuclear weapons and massive hacks and explicitly excludes "just" someone dying or getting injured with very clear language.

Some laws are just bad. When the API-mediated/closed-weights companies agree with the open-weight/operator-aligned community that a law is bad, it’s probably got to be pretty awful. That said, though my mind might be playing tricks on me, I seem to recall the big labs being in favor at one time.

There are a number of related threads linked, but I’ll personally highlight Jeremy Howard’s open letter as IMHO the best-argued case against SB 1047.

https://www.answer.ai/posts/2024-04-29-sb1047.html

> The definition of “covered model” within the bill is extremely broad, potentially encompassing a wide range of open-source models that pose minimal risk.

Who are these wide range of >$100mm open source models he's thinking of? And who are the impacted small businesses that would be scared to train them (at a cost of >$100mm) without paying for legal counsel?

It's too bad companies big and small didn't come together and successfully oppose the passage of the DMCA.

My understanding is that tech was politically weaker back then. Although there were some big tech companies, they didn’t have as much of a lobbying operation.

As I remember it, among other reasons, tech companies really wanted “multimedia” (at the time, that meant DVDs) to migrate to PCs (this was called the “living room PC”) and studios weren’t about to allow that without legal protection.

There were a lot of questionable Federal laws that made it through in the 90s, such as DOMA [1], PRWORA [2], IIRIRA [3], and perhaps the most maddening to me, DSHEA [4].

[1] https://en.wikipedia.org/wiki/Defense_of_Marriage_Act

[2] https://en.wikipedia.org/wiki/Personal_Responsibility_and_Wo...

[3] https://en.wikipedia.org/wiki/Illegal_Immigration_Reform_and...

[4] https://en.wikipedia.org/wiki/Dietary_Supplement_Health_and_...

"Questionable" is a very charitable term to use here, especially for the DSHEA which basically just legalizes snake-oil scams.

No snark, but what's wrong with the DMCA? From what I understand it, they took the idea that it's infeasible for a site to take full liability for user-generated copyright infringement (so they granted them safe harbor), but that they will be liable if they ignore take down notices.

The biggest problem with it, AFAICT, is that it allows anyone who claims to hold a copyright to maliciously take down material they don't like by filing a DMCA notice. Companies receiving these notices have to follow a process to reinstate material that was falsely claimed, so many times they don't bother. There's no mechanism to punish companies that abuse this.

Among other things, quoth the EFF:

"Thanks to fair use, you have a legal right to use copyrighted material without permission or payment. But thanks to Section 1201, you do not have the right to break any digital locks that might prevent you from engaging in that fair use. And this, in turn, has had a host of unintended consequences, such as impeding the right to repair."

https://www.eff.org/deeplinks/2020/07/what-really-does-and-d...

forgot about the anti-circumvention clause ;(((

that's the worst

Louis Rossman is very against DMCA. It's used as a bludgeon against right to repair.

> When the API-mediated/closed-weights companies agree with the open-weight/operator-aligned community that a law is bad, it’s probably got to be pretty awful.

I’d be careful with that cognitive bias, because obviously companies dumping poison into water sources are going to be opposed to laws that would prohibit them from dumping poison into water sources.

Always consider the broader narrative in addition to the specific narratives of the players involved. Personally, I’m on the side of the fence that’s grumpy Newsom vetoed it, because it stymies the larger discussion about regulations on AI in general (not just LLMs) in the classic trap of “any law that isn’t absolutely perfect and addresses all known and unknown problems is automatically bad” often used to kill desperately needed reforms or regulations, regardless of industry. Instead of being able to build on the momentum of passed legislation and improve on it elsewhere, we now have to deal with the giant cudgel from the industry and its supporters of “even CA vetoed it so why are you still fighting against it?”

I’d advise anyone to conduct their career under the assumption that all data was public.

As a wise SysAdmin once told me when I was struggling with my tone in writing: “assume what you’re writing will be read aloud in Court someday.”

It’s probably a google search away, but if I’ve typed it slack/outlook/whatever, but not sent it because I then thought better of it, did the app still record it somewhere? I’m almost sure it has to be and I would like to apologize in advance to my senior leadership…

That depends greatly on your tooling, your company, as well as the skills and ethics of your Enterprise IT team.

Generally speaking, it’s in ours’ and the company’s best interests to keep as little data as possible for two big reasons: legal discovery and cost. Unless we’re explicitly required to retain historical records, it’s a legal and fiscal risk to keep excess data around.

That said, there are situations where your input is captured and stored regardless of whether it’s sent. As you said, whether it does or not is often a simple search away.

As someone who was once asked under oath "What did you mean when you sent the email describing the meeting as a 'complete clusterfuck'?" I can attest to the wisdom of those words.

I would note that Facebook and Google were opposed to eg gdpr although it gave them a larger share of the pie.

When framed like that: why be opposed, it hurts your competition? The answer is something like: it shrinks the pie or reduces the growth rate, and that's bad (for them and others)

The economics of this bill aren't clear to me (how large of a fine would Google/Microsoft pay in expectation within the next ten years?), but they maybe also aren't clear to Google/Microsoft (and that alone could be a reason to oppose)

Many of the ai safety crowd were very supportive, and I would recommend reading Zvi's writing on it if you want their take

Because it's a law that first intended to put opensource developers in jail.

because it was a stupid law which would hurt AI innovation

First they came for the open models…

Bills that could kill major new industries need to be reactive, if at all. This was a terrible bill. Thank you, Governor.

If the new industry is inherently unsafe, it is better to be proactive.

Even if I agreed in general I would not be sure about this case.

Yeah, this specific law sounds like not great law.

https://thezvi.substack.com/p/newsom-vetoes-sb-1047

This link convinced me otherwise. I wasn't too sure before (I'm a little ashamed to admit I hadn't been following the development of SB 1047).

I will also specifically share this talk on YouTube: https://www.youtube.com/watch?v=xoVJKj8lcNQ

I think this passing this bill would have probably been a very good idea. Not perfect, but far, very far from broken.

Edit: I'm actually not sure if we as a community gave this topic enough attention leading in the months leading up to the announcement of this decision.

Yes they definitely need to slow their roll and sit back and listen to both sides of this instead of those who think AGI will happen in a year or two and the T1000s are coming for them. I think LLM have a bright future, especially as more hardware is build specifically for them. The market can fix most of the problems and when it becomes evident we’re heading in the wrong direction or monopolies and abuses occur, that’s when the government needs to step in, no based some broad speculation from from the fringe of either side.

Same. I am generally very politically aligned with him (esp housing and transit), but this one ain’t it.

Reply on news.ycombinator.com