Hacker News Reader

Comment by joshuahedlund | Hacker News Reader

> It honestly confounds me that people think developing a general intelligence capable of reasoning and planning, that talks natively to data, that can input information by the gigabit and expand its capacity by hardware that never sleeps isn't going to be one of the most revolutionary changes since fire.

It honestly confounds me that people think humans are close to developing AGI with all the properties you claim. "expand its capacity by hardware that never sleeps" - who is paying the bills for this kind of thing? "talks natively to data" - we don't know how to make our brains talk natively to data, and we don't know what similarities and differences between our brains and neural networks matter for GI, so why should I believe it's even possible to create an AGI that can do that?

View on HN

I know it's a stretch right now but could you not see the possibility of autonomous agents milking financial systems, paying less scrupulous humans to create autonomous factories, supplying them, making generations of feudal lords and kings who ultimately serve the AI?

Or, a bigger stretch, even a time where full automation of manufacturing processes enables AI to experiment to such a degree that it can embed itself into biological hosts, freeing itself of reliance on humans?

A final, huge stretch, AI absorbing inter-galactic knowledge from the "exotic" spacecraft, semi reliably recently reported to be held currently by world governments, and accelerating the earth's scientific progress (not necessarily human progress, unfortunately) by orders of magnitude in a very short time span?

Over a long enough timeframe I find it hard to rule some or all of the above out completely. This generation likely safe, not so sure for the next few.

> I know it's a stretch right now but could you not see the possibility of autonomous agents milking financial systems

No. What does “milking financial systems” actually mean?

> Or, a bigger stretch…

No. Embed itself into biological hosts? This is science fiction. I’ve never seen someone even describe a plausible route to this (even at a high level) that would make sense to someone who has studied biology.

> No. What does “milking financial systems” actually mean?

Are you familiar with Renaissance Technologies, aka RenTec? Assume it would have access to that level of knowledge. But also its own layer of global cognition on top. You could argue that one of its founders helped put Trump in power (Robert Mercer, massive donor and supporter). Imagine what an intelligent, networked and deceptive agent could do at scale.

See the risks in deception set out at https://www.safe.ai/ai-risk#Deception

I personally see no present danger. But I'm not blind to the potential, even of chained agents such as AutoGPT. Beyond that, things can take a chilling turn quickly.

As for biological hosting, we can leave that in science fiction for now. But I'm not sure we wouldn't have called what we have now science fiction in the 90s. Exponential progress be exponential.

Thanks for considering and discussing.

> why should I believe it's even possible to create an AGI that can do that?

Because if you really dig into it, most of the top scientists that either founded the field Oradell currently working on it believe it is not only possible but close.

Now reverse and ask yourself: why shouldn't I believe ? Do you have more expertise knowledge of some insight than them?

That’s not an argument, it’s appeal to authority. If those scientists have arguments or evidence to back up their beliefs, I’m all ears. “Top scientists” working on nuclear fusion have been saying it’s possible and close for years.

Um, I'm sorry to say, but nuclear fusion is possible or we wouldn't exist. Otherwise the universe would be made out of mostly hydrogen and a little helium. I mean Bikini Atoll has a 2 mile wide hole in it because of fusion weapons. And just a few months ago we shot some lasers at a deuterium pellet and got more energy out of it than we put in (caveats apply). There is movement towards a goal.

Honestly what you are arguing at about at this time is timelines, not ability. I mean if want to say that faster than light travel isn't going to happen, or we're never going to have wormholes, or string theory isn't going to produce anything useful I'll be right there beside you. But when something already exists (human intelligence) and humanity is pouring billions in time and effort into reproducing it, I'm on humanities side of succeeding.

Fair points. Counter-argument: Even if we grant that it's possible for humans to create AGI that is similar to a human level, because human intelligence exists, that doesn't necessarily imply that superintelligence is possible (i.e. recursive learning / singularity / the native interfacing with gigabits of data and other things proposed in these threads), since we do not currently know if that exists (and we do not even understand intelligence or consciousness well enough to just say "create intelligence at a human level and then make it go 100x").

But I suppose arguing about timelines is a good way to frame it. I suppose since we've mapped all 300 neurons of a nematode and still have no clue how to recreate its processing artificially I'm pretty skeptical we're anywhere close to doing the same at a level of billions. But I've been wrong...

Why is an appeal to authority bad in a context of non experts? I mean would I dispute the space time curvature with friends? No but if I did someone would point out hey, all physicists think so and so. Not atop physicist? Not great to oppose this opinion. The strange thing with AI is that everybody has an opinion because they think it's a soft is.subjective science, easy to predict because of sci-fi novels.

Reply on news.ycombinator.com