Hacker News Reader

Comment by svara | Hacker News Reader

I'm mostly a fan of AI coding tools, but I think you're basically right about this.

I think we'll see more specialized models for narrow tasks (think AlphaFold for other challenges in drug discovery, for example) as well, but those will feel like individual, costly, high impact discoveries rather than just generic "AI".

Our world is human-shaped and ultimately people who talk of "AGI" secretly mean an artificial human.

I believe that "intelligence", the way the word is actually used by people, really just means "skillful information processing in pursuit of individual human desires".

As such, it will never be "solved" in any other way than to build an artificial human.

View on HN

No, when you bring in the genetic algorithm (something LLM AI can be adjacent to by the scale of information it deals in) you can go beyond human intelligence. I work with GA coding tools pretty regularly. Instead of prompting it becomes all about devising ingenious fitness functions, while not having to care if they're contradictory.

If superhuman intelligence is solved it'll be in the form of building a more healthy society (or, if you like, a society that can outcompete other societies). We've already seen this sort of thing by accident and we're currently seeing extensive efforts to attack and undermine societies through exploiting human intelligence.

To a genetic algorithm techie that is actually one way to spur the algorithm to making better societies, not worse ones: challenge it harder. I guess we'll see if that translates to life out here in the wild, because the challenge is real.

The troubling thing here is: what is a "better" society? As you said, it's just the one that outcompetes the other societies on the globe. We'd like to believe such a thing is an egalitarian "healthy" liberal society, but it's just as likely to be some form of enslaved/boot stomping on face society. Some think people won't accept this, but given human history I'm pretty sure they will. I think these sorts of societies are more of a local minima, but they only need to grant enough of a short term boost to unseat the other major powers. Once competition is out of the way they'll probably survive as a bloated mess for quite some time. The price of entry is so high they won't have to worry about being unseated by competition unless they really screw the pooch. I think this is the troubling conclusion a lot of people, including those in power, are reaching.

I tend to agree.

I'm certain neanderthals were just calmer, more empathetic. And then we came along and abused that until they were all gone.

We're still animals on this planet. We just sing about our conquests afterwards.

It's worth thinking about, but why hasn't this already happened? Or maybe it already has, and if so, what about AI specifically is it that will make it suddenly much worse?

We've had plenty of examples of all those things, over and over, throughout history. Nothing's really new. Societies that get into faceboot territory run afoul of what's already known (there's apparently a CIA handbook to this effect that's being largely ignored in modern America): assert hard rather than soft power and you generate determined and desperate resistance more than you undermine it. That's being demonstrated in countless places right now.

I'm arguing that the egalatarian 'lift my lamp beside the golden door' society is a cheat code for producing the variety and ferment that makes everybody frustrated and unhappy but producing with wild abandon. As a society this tactic dominates the hell out of would-be ethnostates and dictatorships, which seems to also be a natural tendency of humans. They are interested in not being challenged, in those like them not being challenged. Comfortable for those fortunate individuals, hopelessly suboptimal for the society they're in.

The rallying cry of 'NO New York Cities! Only sundown towns where if you don't look right you are killed and nobody ever knows about it!' might please some people (who have never been anywhere near those evil cities) but it just goes to show that many people have unhealthy wishes that are bad for them and the societies they're in.

> If superhuman intelligence is solved it'll be in the form of building a more healthy society (or, if you like, a society that can outcompete other societies).

Maybe so, but the point I'm trying to make is this needs to look nothing like sci-fi ASI fantasies, or rather, it won't look and feel like that before we get the humanoid AI robots that the GP mentioned.

You can have humans or human institutions using more or less specialized tools that together enable the system to act much more intelligently.

There doesn't need to be a single system that individually behaves like a god - that's a misconception that comes from believing that intelligence is something like a computational soul, where if you just have more of it you'll eventually end up with a demigod.

Reply on news.ycombinator.com