Hacker News Reader

Comment by sterlind | Hacker News Reader

I think it's likely we'll develop superintelligence very soon, but it won't be any more knowledgeable than us, just smarter. And knowledge has been the bottleneck for most of human history - our ancestors weren't dumber. Also, it will be bottlenecked by needing to solve NP-complete problems - unless P=NP, it's not going to be all that much faster at solving hard math problems. Basically, it still needs to do science, and science takes time.

View on HN

Take a smart human and speed up their thinking a billion times, you get the time for free. What takes humanity a decade of effort could happen in a day (or faster) that's the nature of thinking faster. People often mistakenly model a smart human compared to a dumb human, but the actual spectrum of capability is way wider than that.

Lack of knowledge could be another way of saying "not aligned" which is the core issue.

that's the thing though, I don't believe superintelligence can possibly work like that. I think it's a logical fallacy to treat superintelligences like hypercomputers. they're constrained by existing on classical computers.

like, as a trivial counterexample, I can speak at 500 wpm. let's say that ChatGPT can generate 500 words per second for a single thread of conversation - I think that's a generous overestimate. now that's a 60x speedup over me, not a 1,000,000,000x like you're talking about. do you honestly believe they they can make ChatGPT run 16.6 million times faster, without changing hardware? do you think ChatGPT will just like, hit an inflection point where it realizes how to refactor its inference code to run 16,666,666x faster?

no, I think that's absurd. you're treating these things like black boxes but they are constrained by computational complexity, die area and the speed of light for Christ's sake.

By that logic, we should have never broken the sound barrier because we're under the same physics constraints as other biological organisms. The key difference is that this new intelligence is not constrained by having to slowly pump ions across cell membranes in order to internally generate power for computation.

Silicon transistors are much more efficient at computation than the human brain just as a jet engine is superior to to a peregrine falcon.

ChatGPT could absolutely generate thousands of words per second on existing hardware.

The comparison shouldn't be to individual humans, but to societies of all scales.

AIs can scale horizontally as well as vertically.

What is the limit of intelligence? What limits the human intelligence? How many more times intelligent would an AGI have to be to have the capacity to take over the world?

GPT-4 speaks every human language, knows every programming language, and can answer introductory and sometimes even advanced questions in history, law, biology, and mathematics. In what way is it not more knowledgable than us?

Yes, everything there is already known by someone. But look at medicine. Specialists who are able to recognize conditions and recommend treatments better than others make fortunes, sometimes just for briefly looking at patients and answering other doctors' questions. Look at cybersecurity. A lot of exploits come from knowing something the victim didn't about a lot of different pieces of software or processes, chained together. Being able to think through the whole of human knowledge, or even the whole of a single field like biology, is something no human can do.

Also, GPT-4 is an existing system, one inconceivable a couple of years ago.

Reply on news.ycombinator.com