Hacker News Reader

Comment by krageon | Hacker News Reader

Quite an interesting paper in it's impact study, but the questions it asks all boil down to "the masses are not to be trusted with our vaunted knowledge, because harm can be done. How can we prevent them from moving outside the boundaries of knowledge we like" and then the example used is how GPT-3 is managed. This is without a doubt a terrific example of bad technological stewardship: Any malicious state actor gets to use what you make for evil, while at the same time keeping it away from the folks you're making it for. I wish they had had a more substantial and innovative thing to say about it, but as it stands it reads as regressive.

View on HN

I disagree with your interpretation.

Would it be better if nobody had access to easier ways to produce new and innovative nerve agents? Yes, of course it would be better.

Do I prefer a world where only state actors can build nerve agents, vs a world that includes state actors plus more? Yes, less sources of nerve agents is more good.

You might as well complain how pointless it is to block proliferation of nuclear weapons when malicious state actors already have them.

The world wont be safer by spreading nuclear weapons to more people, and similarly for nerve agents.

Fortunately, this sounds similar to nuclear weapons in another way, designing the weapon is not the hard part that limits them to state actors, its producing them that's hard.

Still, nuclear powers don't tend to publish their weapon plans publicly, for good reason, and the same logic should apply here.

I'm not using the comparison to imply this is as serious as nuclear proliferation btw, just that it's a useful example of knowledge that really should be kept limited.

> "the masses are not to be trusted with our vaunted knowledge ..."

The masses will use AI to do bioweapon design? That doesn't sound like something most people could do.

Most people also wouldn’t shoot up a school, yet here we are

Knowledge and resources necessary to shoot up a school: location of dad's AR-15, bullets, and how to load.

Knowledge and resources necessary to develop AI-proposed toxins: multiple PhDs, thousands of square feet (probably 10s to low 100s of thousands) of lab space, including nasty stuff, like bromides and heavy metals, and testing and evaluation of various delivery vehicles, ranging from canes with needles to air-launched warheads with atomizers.

My point is that the 'masses' have no interest in this knowledge.

But ..

"Risk of misuse The thought had never previously struck us. We were vaguely aware of security concerns around work with pathogens or toxic chemicals, but that did not relate to us; we primarily operate in a virtual setting."!!!

Operating in the rarefied air of 'science' they were unaware of the risk of misuse!

I don't think they can even conceive of the idea that the government might misuse this technology.

They wrote a paper on it. They weren't just aware, they were demonstrably aware. Or are you suggesting that we blame people for having a finite mental processing speed?

I'm saying its a funny to say that they were unaware of the risks when they started on the research. I mean, its not beyond imagination that AI could be/is being used for nefarious ends.

I think we agree...??

> Operation in the rarefied air of 'science' they were unaware of the risk of misuse!

The thundering irony of this concept in a timeline where a (group of) scientist(s) discovered fission cannot be overstated.

The thundering irony of blaming the group of people who made you aware of a problem for being unaware of the problem is even more difficult to overstate.

Do you think everybody should have the right to design a Sars-Cov-4 in their kitchen just because it's soon going to be technologically possible and free speech?

Reply on news.ycombinator.com