Hacker News Reader

Comment by topspin | Hacker News Reader

> "Stop" is not regulatory capture.

"Stop" is a fiction that exists exclusively inside your head. Unless your solution anticipates a planet wide GULAG system run by a global government your "stop" is infeasible.

View on HN

His solution -- assuming it's the same as the one proposed by Yudkowsky, who he quotes -- is launching air strikes against data centers hosting AI computation.

So yeah, imaginary.

Ironically, it seems the only way to stop someone from building an omnipotent AI that could destroy the world is to build an omnipotent AI under your control faster than any of the people you need to destroy. If you believe in that sort of thing.

The only thing that can stop a bad guy with AI is a good guy with AI.

AI's for everyone!!!! It's actually a scare but possible future. Everyone has to carry personal security AI that you rent from some Corpo, and if you don't you will be endlessly hacked and harassed in IRL just like we are online these days.

Heh, sounds like we're reading Diamond Age again.

Or possibly just not allow anyone to accumulate enough compute power, given that even the largest model so far still isn't that.

Hence "if you're not willing to go as far as air strikes, you were never serious" or however he phrased it.

This is just the plot of War Games (1983).

I wonder what would happen if the AI computation is co-hosted in data centers with civilian data.

Then people would have their claims of "bulletproof hosting" and offsite backup policies/business continuity plans tested.

Yukowsky is presumably a "we have to kill the humans in order to save them" maximalist.

Ah, so we're going to start the global nuclear exchange with China based on the paranoid fantasies of some guy on Twitter? And this is supposed to be taken seriously?

Well, you're reacting to a HN user paraphrasing what seems to be a single sentence taken out of context of the actual argument. Did you expect it to make sense?

And world war two was started by some failed artist from Germany. Deliberately reducing things is easy and does not prove anything.

That might be true but there's no way of knowing for sure. Planet wide gulag system(a colorful and a bit wrong name for prisons) already exists and is pretty aligned on topics like killing fellow humans as a result of conscious, deliberate action. I find Max Tegmark's argument pretty convincing. We seem to agree to "STOP" on things like genetically modifying ourselves at conception or birth or whatever to gain X or Y and the one loud case of violating that agreement came from China and resulted in a ticket to the gulag. Just ask chatgpt to give you examples of fields where further research was deemed too risky and was ceased.

Unlike for AI, in those fields we do not need to stack endless hypotheses on top of each other to arrive at something bad (and even then some prohibitions are tenuous in their reasoning at best).

I don't know about that. Cloning/modifying humans has potentially so much benefit and yet we seem to stack hypotheses on top of it and conclude it's not worth it. There is also a lot to be said about all the hypotheses of killing all the Hitlers preemptively. I think that whatever we do, we usually arrive at a bit modified trolley problem and try to go from there. We really should never expect that the one particular route that we choose carries all the benefits and the other all the downsides.

My point was more that it really isn't stretching anything to say: if something bad gets into human reproduction that has unfortunate effects. We can observe that already. So at least the risk case doesn't rely on a lot of hypotheses that might turn out wrong.

Whether or not all regulations there are good or optimal is then a separate point again.

I agree but this is too general. If something bad gets into AI (like very biased or false training data) that has unfortunate effects also stands.

But we need to stack a bunch of hypotheses on top of each other to have dangerous AI and those might not be true - that is the difference.

The best argument against human cloning/gmo is simply that we can't do this well on other animal species just yet, so probably we should be hold off on the humans. It's hyper pragmatic from an ethics perspective, not a trolley problem or stack of hypotheses at all.

I'm no expert but I actually think we are pretty good at genetically modifying organisms and that manifests itself in crops for instance with enough research to support the benefits.

Plants are one thing we do it well enough on. As far as I'm aware we have yet to do it (at scale) on livestock, or even pets.

There is: https://en.wiktionary.org/wiki/Enviropig or https://en.wikipedia.org/wiki/AquAdvantage_salmon

or things to the likes of https://www.avma.org/news/genetically-modified-cattle-may-be...

Looks like the fish is the only one that's gone to market. Pretty clearly still early days. I have no doubt that with enough time we'll get there, but to say this is proof that we're technically good to go ahead and produce gmo humans seems a bit rushed -- to reiterate, the point being that the ethical issues here are much more clear than a trolley problem or stack of hypotheses.

I think much of this is, because Monsanto aside, no one has had a nice big genetic accident or intentional 'oh no' yet. Yea, we've had issues but not at the people are starving.

If we start thinking of plant based biological weapons, just imagine modifying an existing grass to produce tons of pollen and to cross pollinate wheat but not produce seed in the wheat.

"Stop" might be possible if doing AI is expensive enough like say biotech. Currently it is in the limit of being expensive(costing few hundred thousands to few million for state of the art model), but it is very likely to change in near future that will change as even currently most of the cost comes not from energy of computation, but from expensive NVIDIA hardware.

If tomorrow TSMC, NVIDIA and ASML get destroyed in the next 9/11, it's "stopped". It's completely possible.

It's delayed at best.

Incredibly delayed, giving at least decades to figure things out.

Fantasies, bordering on threats.

Of course it's a threat. Civilians doing that would be bad. But governments do much the same via military intervention on a regular basis, hence Yudkowsky talking about airstrikes that one time.

I'm not suggesting to do that, I'm just showing that it's incredibly possible. You could also go with "Tomorrow a massive storm strikes the Netherlands and huge earthquakes striking Taiwan and the US Westcoast", if you have emotional objections to my first example.

I think current hardware already produced till now is either enough to get us to AGI, or it is very far away. I don't believe AGI is 30 years away. It is either very near or more than 1000s of years away.

The sum of all hardware produced, sure. But the amount of hardware a single player currently posseses? Doubtful.

Reply on news.ycombinator.com