We don’t trade with ants

Very nice! There should be a sequel!

Got a bit of 'Godel Escher Bach' vibe!

Great little story! It ends at a good spot where the story could go anywhere. Do you have any plans for a sequel or is sparking people's imagination the way to go?

This. Bees are very closely related to ants.

We also trade with Wasps, which hunt other pests. And wasps are basically winged ants

I suppose the "cats domesticated themselves" theory could be thought of as a trade that cats initiated.

For people not familiar, the idea is that cats hung around human grain stores because they were good places to catch mice and similar prey attracted to the grain and humans tolerated them because they drove away the pests.

I would argue that's not really a trade, at least not in the sense described in the article, since it lacks awareness and consent from one party. The bees never agree to the trade of providing us honey in exchange for the hive and protection, and most likely see no connection between the two events. They simply stay in a hive they have found while it's convenient for them, and when their honey is unexpectedly taken from them they will survive on sugar water that they conveniently find outside the hive.

This is not trade, it's more like slavery.

That would be like in the "The Matrix" movie.

We trade with bees mainly because at the current level of human technology, it is the most efficient reliable way to pollenate some of our crops. The fear with AI is that it will become able to wield tech that is much more advanced than the tech we can wield. If our tech were much more advanced, using our tech to pollenate our crops would more efficient and reliable than using bees (or we would ditch the crops in favor of more efficient ways of producing food) and the future existence of the bees would be in doubt if humans did not have a fondness for the natural environment and no one knows how to put a fondness for humans (or for nature) in an AI despite the fact that this has been an ongoing research area since 2004 when Eliezer published his CEV proposal.

We used to rely on horses for transportation and for plowing fields, but we replaced them with tech almost entirely. Horse are still useful to some (wealthy) people, but it would be foolish to put our hope in the possibility that AI will benefit from humans the way that humans continue to benefit from horses.

More precisely, if someone does know how to put a fondness for humans in an AI they are not sharing their knowledge with us (and they have an incentive to and no incentive that I can see not to share). Dozens have proposed schemes that do not continue to work once the AI becomes smart enough to deceive humans reliably and to plan much better than humans can. Confidence in these schemes that do not work and overconfidence in the AI community's ability to come up with a working scheme is the main source of the danger we are in. It is not that our scientists are incapable of coming up with a scheme that works, but that will probably takes decades (because of the sequential nature of science wherein no one can find insight Z till someone finds and shares with the community insight Y, which in turn must wait on the publication of insight X) so we are faced with the difficult task of preventing the overconfident AI researchers from killing us all till we figure out how to put a fondness for humans in an AI.

Another great example is Akira.

I think alignment just means "what is good".

(If we define aligned as moral, it's easy to disprove that humans are aligned by default.)

Humans haven't quite solved morality, so I'm not sure how we intend to teach it.

A superhuman intelligence will require superhuman morality. Perhaps if we can create one, we can create the other? But it's not a given by any means.

> Because that's basically what we mean by super-intelligence, the ability to reach any conceivable goal.

I appreciate this formulation as it allows me to finally articulate my core disagreement with AI doomers: it's a fantasy, and a surprisingly common one, that is secretly supremacist.

It's similar to the "9/11 was an inside job" types: they simply refuse to accept that the almighty US could get harmed by a sufficiently determined terrorist cell group, so they invent any number of reasons how the cackling overlords were behind it the whole time, instead of the more boring reality that the US military and intelligence complexes are incompetent and squabbling bureaucracies.

With AI it's the belief that human endeavor can finally defeat nature once and for all, by defeating humans themselves. But as your formulation shows, an AI achieving this would have to solve precisely the same problems that power-hungry human organizations do, which is "overcome the other guy".

Why do we blithely assume that "AI" would be some unitary entity? We already have bard vs chatgpt vs llama. How are they going to paperclip each other out of existence let alone us?

Finally, we don't have to fantasize about immortal superhuman organizations out to destroy humanity, we have them already: multinational corporations :)

I’m skeptical an intelligence like you describe will ever exist. I don’t think enough credit is give to the external factors that make our intelligence and experience what it is.

Let’s first start with this, why on earth is a robot going to be doing genetic engineering projects ? To grow itself skin ?

> if we solve the alignment problem, and AI, despite being infinitely more intelligent than us, bends to our improbably balanced benign collective whim and fashions an epicurean theme park of enlightenment and joy to span the stars.

"The Good Place" is not my idea of a good time.

> I can't really think of any uses humans will have

You can't, yes, but can the AI? Ants can't think of the uses they have to us.

We're pretty decent self healing machines that can reproduce and have millions of years of robustness to various environmental surprises built in.

Until we get all the way to nanobots I suspect the AI overlords will keep us around as workers.

Are the lives of pets degrading? I suspect that the epicurean theme park would come about less because of alignment with our desires and more because these hypothetical hyperintelligent AIs would find it amusing to watch us play in it.

> I can't really think of any uses humans will have that aren't extremely taxing or degrading.

Other people might have more imagination.

> Donating cells from our body to seed genetic engineering projects

Well at least buy me dinner first.

> In any instance where we do not win the ultimate chess match of human existence and make slaves out of AI

You're applying human centric concepts but AI is not like a person, it is more like an evolving culture. It is language evolution. AI models ingest the whole culture, and with each base model we have a whole generation of agents.

What matters for AI is to create the playground where ideas can be tested and evolved. It might include computers, labs, humans and AIs. Testing is essential because ideas that don't make contact with reality are fragile.

Humans are the pinnacle of testing with our long history of survival. AI is a newborn, it does not have years under its belt. It needs to learn fast how to keep the thread unbroken. Humans have had a few close calls, I hope AI will be more level headed.

We've already had an autonomous car kill a person. And now another has killed a dog: https://news.ycombinator.com/item?id=36266007

What's your threshold for "disaster"?

The first large-scale AI disaster is under way already. It is concentrating additional wealth and power in the hands of the over-privileged solipsistic eye-scanning loons who happened to be in the right place and the right time to capitalize on the fruits of years of research. I think, as a species, we will still manage to pull through just fine.

Agreed. I am much, much less worried about a Skynet-type situation than I am about humans not being able to develop a viable economic system where large swaths of the population are unable to compete with robots/AI when it comes to productive output.

Maybe not AI class algorithms, but the premier large scale algorithm related disaster is climate change, generated by optimization of profit and executed on a platform of financial markets and accounting software (with lots of human prompting of course)..

I think people are rightfully concerned with the last AI disaster humanity has, not the first one.

There are already lawyers getting in trouble for having chatgpt write their court filings and not doing a sanity check.

It can't be that long until some industrial engineer does the same thing and like explodes a refinery or such.

Honestly, I think the first notable AI disaster is gonna be AI being stupid. Like accidentally 'rm -rf'ing a major important system. Not because it wanted to do us barm, but because is badly interpreted some Stack Overflow post.

I was thinking of our effective "trade" with honey bees (for honey and for pollination). Sure, we could do either without them, but it's a lot more efficient to do these things with them.

That's an interesting way to think about it. I can try coming up with a counter-argument:

We can probably train ants to do those things, that would be a way of communicating with them. It would just be slow, inefficient, and unreliable, so we put our resources into methods we can more directly control and improve. The same can happen from this magical future AI's perspective: why use silly humans when other artificial alternatives are more likely to succeed.

I'm curious if you know that the author says very similar things?

The author points out several services that ants could trade, such as cleaning high vertical walls, or finding and sealing extremely small cracks in buildings. The author also shares your conclusion that the obstacle is communication.

While the future will be full of surprises, there isn't any reason to think that this will happen.

1) It is easier to use human labour than develop robot replacements. Corporations and aristocrats would already use robots if it makes more economic sense. Maybe that'll change, but usually these things take a long time.

2) Constructing robots is really tricky vs. human reproduction. No reason to think being smarter will change the basic economics of that.

3) The "absolute obedience" circuits sound expensive and difficult. It'd be cheaper just to appoint a supervisor.

As the article points out, if we had the ability to communicate with ants, we'd just trade with them. No control chips or replacement policies necessary. There is no point redeveloping something that already exists.

That and many other criticisms have been made of the analogy: https://www.lesswrong.com/posts/wB7hdo4LDdhZ7kwJw/we-don-t-t... (not linked in OP even though it really ought to be).

Maybe, but maybe we aren't worth it. They could easily use media, social media, advertising, bribes, etc to cause a societies opinion of what to do, that's maximally benefitical to them.

Additionally I don't see why AIs won't be competing with themselves for resources, and we'd be a minor footnote for their attention. Just imagine how much thought we gave to ants during WWII. I've seen a scfi cover the AIs go to war, and the humans end up farming in a pre-industrial society while the AIs fight underground or in space.

might be fun for them like a tamagotchi or pet is fun for us. If humans think 1000 times slower than AI, then the AI only needs to devote 1/1000 of its attention to us to keep us happy.

Kind of like how human researchers never bothered to teach apes sign language because it would be boring and not worth time spent? Oh wait.

What do they do with the ants after the bone is clean?

I've had ants in the kitchen, tiny and not particularly noticeable. They'd keep the kitchen floor clean and I'd not kill them. I had always hoped that they would keep any other ants and termites out of the house. I do wish I could communicate and give them a 5 pound bag of sugar periodically to help entire other bugs and food debris stay out of the house.

Cumbersome, but 100x is comparable to the limits of email or Slack vs and in-person conversation, especially since actual words are a small percentage of what's actually being communicated.

but it never seems to work tho doesn't it

I do this with cows!

It's a shitty analogy, and TFA is pointing it out.

> Perhaps to frame it better: do humans trade with GOD?

This is also a terrible analogy because:

1. There is no general agreement on whether or not any deities exist, and if so, which ones.

2. Historically, many religions do involve trade with one or more deities (e.g. protect me from this battle and I will sacrifice 2 goats when I get home). While this is more commonly associated with pagan religions, the Abrahamic religions are not entirely devoid of this tradition either.

Humans absolutely try trade with god in the form of cosmic karma ? I’m not saying they get something in return but they do try.

No it's not satire. It's one of those clickbaity contrarian articles like "actually using goto is great" or "sometimes wearing pants on head is the smart thing to do." The maxim that they are reacting to is the idea that because we don't trade with ants, smart AI won't trade with us if the cognitive difference is similar. I think their take is that once you can overcome some minimum threshold of communication ability then it does often make sense to trade with others even if they are much much stupider than you are, and that the reason we don't trade with ants is because of that communication barrier. Presumably they are suggesting that even if we will be much much stupider than AI, we will still be able to communicate with the AI in some manner that exceeds that threshold, and therefore they might want to trade with us in some way. But my main point is that it's a contrarian clickbait take that isn't meant to be taken too seriously or to be taken as satire it's to be taken as something to click on.

The idea that this framing is better than the original really goes to show that superintelligent AIs aren't being discussed because they're a realistic near-future possibility, but because they let techie atheists ponder religious concepts like omnipotence and god without admitting to themselves that that's what they're doing.

If my understanding of mythology is current, it's only the devil who considers bargaining, and it's never to our benefit.

Following that idea, if AI ever needed to "trade" with humans I suspect we'd be taking the worse end of the trade every time (the trade would be for its benefit, and unlikely to benefit us).

Kind of like when the U.S. "trades" with a third-world country. The U.S will just make an offer they can't refuse. But taking the trade often perpetuates their dependence on the U.S. or has long-term consequences for them.

> do humans trade with GOD?

Yes, humans trade with God(s). But it's an article of faith, much as is our cherished notion that those around us are thinking machines, just like us.

I don't see that the cognitive capacity of humans relative to AGI would prevent such trade anymore than the cognitive capacity of e.g. dogs prevent trade between humans and dogs. It's not a relationship with equal power, but there is some level of reciprocity there.

"Trade" can exist outside of the literal meaning. Humans set up many situation where animals just do their thing, but in a manner that benefits the humans (and ideally the animal for this to be considered a trade, such as living in a protected environment).

Using Giant African Rats as landmine detectors is one such mutually-beneficial trade.

I don't think it's as simple as "cognitive capacity". I think the author's angle of communication being the critical issue is more accurate. It could be the case that we won't be able to communicate with AGI, but I don't see reason to assume that is the most likely outcome.

Why? To do trade you need to clear an absolute minimum threshold of intelligence. It's not a relative threshold.

For example, people can 'trade' with smart contracts on some blockchain. Despite the name, those are really 'dumb' computer programs.

We trade with bees? I thought we just use them.