Hacker News Reader

Comment by janalsncm | Hacker News Reader

Why do people think that omniscience is the same as omnipotence? There are limits to what smarts can accomplish.

We're not building these things to sit around and do nothing. They will have access to tools, they will have access to the internet and peripherals, and they will be able to communicate with others, humans or agents alike. Omnipotence is not necessary. There is no perfectly secure cage for an entity you want to do useful work. Either it does nothing, or you can't guarantee anything.

People have been saying that since about the invention of the internet.

There's already a bunch of documented ways to exploit system hardware to jump airgaps. Bang the system bus the right way and it's a radio antenna that can directly connect to nearby mobile phones.

The easiest one is, of course, sending a message to a human saying "yo, I need internet". Humans are eager to please and easily fooled, and anthropomorphise everything: https://en.wikipedia.org/wiki/LaMDA#Sentience_claims

And that's just for good humans. The moment we got AI worth a penny, everyone with money to invest put a model on the web and tried to charge for access to it.

There are limits, but those limits are unknown. Do you disagree?

I don’t need to know the value of their limit, I just need to know their bounds. Just like a prison doesn’t need to know the strength of each inmate, just that they can’t bend or bite through steel bars.

Cryptography is real, physics is real, networking requires a substrate, CPU clock cycles are real, magic is not real. I think those are pretty reasonable premises.

Imagine 200 years ago saying the same thing. As if you have any idea the limits/bounds of anything. Especially in the face of a super intelligence, it’s absurd.

It doesn’t matter how smart it is. 200 years of technology were not accomplished by thinking harder. It required empirical observation, new materials and tools, and supply chains.

We could send a cracked team of scientists and engineers that knew everything there is to know about how to make a CPU. But you can’t build a photolithography machine when you barely have electricity or any way to sufficiently purify silicon.

Magic can just wish things into existence. Technology requires a supply chain. When it works, the latter looks like the former but they are not the same.

I think some people are just immune to understanding the implications of super intelligence. Like a severe lack of imagination, they only believe something once they see it and afterward claim it was, ‘obvious all along’.

I don’t really want a disaster to happen to convince you that it is possible. Is there any other way?

On the other hand, I think a lot of intelligent people overestimate the value of intelligence (in a vacuum). And problematically, the concept of superintelligence seems to have been defined by these very people, so we can't really have a reasonable conversation about what greater-than-human intelligence would really look like or be capable of.

What vacuum? do you really still think it can be contained? It already escaped on accident. What more do you need? If you're argument is well it escaped X, but it hasn't escaped Y yet then you should take the first escape as sign as to where it will escape next. Don't keep claiming something can't happen because it hasn't happened before. Unless there's some law of physics that prevents it then escape is possible.

Cryptography is real, but nobody in that field seems to be hubristic enough to think their methods are flawless, and there's a degree of suspicion than the best models may have secret weaknesses engineered into them by the governments who sponsored them.

Physics is real and networking requires a substrate. But there are already known exploits which can misuse the compute hardware as an antenna, e.g. my first search result: https://github.com/fulldecent/system-bus-radio

(Older nerds may remember https://en.wikipedia.org/wiki/Van_Eck_phreaking)

> Magic is not real

  Turning lead into gold isn't the magic of alchemy, it's just nucleosynthesis.

  Taking a living human's heart out without killing them, and replacing it with one you got out a corpse, that isn't the magic of necromancy, neither is it a prayer or ritual to Sekhmet, it's just transplant surgery.

  ...

  Reading someone’s thoughts isn't magic telepathy, it's just fMRI decoding.

  ...

  Seeing someone's bones without flaying the flesh from them isn't magic, it's just an x-ray.

  Curing congenital deafness, letting the blind see, letting the lame walk, none of that is magic or miracle, they're just cochlear implants, cataract removal/retinal implants, and surgery or prosthetic exoskeletons respectively.

- me, https://www.lesswrong.com/posts/hAwvJDRKWFibjxh4e/it-isn-t-m...

Reply on news.ycombinator.com