Comment by simoncion | Hacker News Reader

Comment by simoncion | Hacker News Reader

Parent Full thread

> I think we need strong action now...

You and I and Nvidia CEO Jensen Huang seem to agree on this. Excerpts from his interview with Ezra Klein: [0]

Klein:

  But what I hear the various people in the lab saying is: We are in this. We feel we are losing control of what we are creating. We want help to slow down where it’s not a collective action problem.
  
  So why are you resistant to that?

Huang:

  Because these are companies with agency. These are C.E.O.s with agency. ... They could absolutely take care of the situation.
  
  Ezra, it’s so weird. If a car company, competing with a bunch of other car companies, which they are — I’m competing with all kinds of companies, which I am. If I believe that I’m about to launch a product that is unsafe, it is completely in my ability, my power and my responsibility, and I’m incentivized to do so, to not launch the product.
  
  And so I can’t buy into the idea that somehow, all of Americans, around 400 million of us, are pushing them to launch untested products that are unreliable, engineered poorly, because they thought they were trying to help us. Don’t do it for me, OK?
  
  And therefore, I think we’ve got to break it down. I mean, it’s really, really serious.
  
  The fact of the matter is, there are so many laws, there are so many obligations, they’re so incentivized to ship safe products. If they ship unsafe products, their customers go away. If they ship unsafe products and they harm somebody, they could have a civil lawsuit. If they ship something and they did it knowingly, there could be negligence involved. There could be criminal lawsuits.
  
  The fact of the matter is, there are plenty of incentives for them to do it right. So I have to disagree with your premise that somehow somebody’s pushing them to do this. Nobody’s pushing them to do this. ... I’m saying that we have lots of laws and regulations. Apply it.

Former FTC chair Lina Kahn has suggestions, too. [1]

Thoughts?

[0] <https://www.nytimes.com/2026/09/23/opinion/ezra-klein-podcas...>

[1] <https://x.com/linamkhan/status/2099204390548639960>

I agree that the weight of the law should be brought to bear on OpenAI. Ideally they should be brought in to testify before congress as well. I'm not a lawyer so I can't really comment on whether current laws are sufficient, but I believe that there should be laws specifically governing the development of AI, with a requirement that a given architecture and reinforcement mechanism be _proven safe_ before training begins.

> ...I believe that there should be laws specifically governing the development of AI...

Why? Existing truth-in-advertising, liability, safety, and -where and when appropriate- weapons-development laws and regulations constrain the past and current conduct of the LLM manufacturers just fine.

The only possible reason for making new laws that I can see [0] is that existing laws "don't work" because the LLM manufacturers are ignoring them. Which, like, _if_ the new laws are going to actually constrain their behavior, why the hell would the LLM manufacturers pay any attention to them? They've already demonstrated that they give zero shits about the existing laws that prohibit what they have been doing and continue to do.

[0] ...that isn't "The LLM manufacturers are engineering a panic with their very real, actual, and actually alarming conduct so that they can 'guide' lawmakers and regulators into 'accidentally' letting the LLM manufactures capture those who would regulate their behavior"...

Liability only kicks in after damage has been done. As far as I know, there is no law that could currently force AI companies to only test cybersecurity capabilities in air-gapped datacenters, for example; only laws that could punish them if their cyber testing led to a hack that caused material damage. But if, lets say, a rogue AI agent swarm attacked a hospital and caused patients to die, no amount of liability will bring those patients back to life.

> Liability only kicks in after damage has been done.

a) Both OpenAI and Anthropic have done far more damage with their jaw-droppingly-sloppy testing of computer-attacking tools than Aaron Swartz did by downloading documents from JSTOR. It's good to see that you and I both agree that there are things for them to be prosecuted for.

b) Is your claim that the cost to thoroughly investigate and clean up after a cyberattack doesn't count as damage? If so, that runs contrary to every relevant claim of damages in a CFAA case that I've seen.

No I absolutely think they should be prosecuted, and at a minimum owe damages to all of the companies that their agents hacked.

What I'm saying is that that is not sufficient to stop future harm; I expect the total damages would be less than the cost of a full training run, so it would effectively just be the cost of doing business. Liability is not sufficient to protect the world from dangerous technology - we need proactive rules around how the technology is developed, tested, monitored, and deployed, as we do with other dangerous industries such as airplanes, nuclear reactors, weapons manufacturers, etc

> Liability is not sufficient to protect the world from dangerous technology - we need [new] proactive rules...

You and I couldn't disagree more.

The major LLM manufacturers are begging for new laws and regulations so that they get a huge hand in writing them. Regulatory capture is absolutely their goal. Given that they claim to believe that they're working on WMDs [0] that they cannot adequately control, they'd just stop work if safety was their goal. Their collective cries for regulation demonstrate that they'll happily coordinate with each other if they think the issue is important enough to do so. I guess "preventing the extinction of the human race by way of weapons we built and let slip from our hands" isn't sufficiently important.

[0] See the second paragraph and associated footnote here for a justification for the use of this term: <https://news.ycombinator.com/item?id=49839682>

Reply on news.ycombinator.com