Comment by DanielBMarkham | Hacker News Reader

Comment by DanielBMarkham | Hacker News Reader

Parent Full thread

If you'd like to join me in speculating a bit, let's take an imaginary look at ten years from now.

Good news! AI has advanced enough that it is now possible for you to have a customized bot that handles all of the crappy and mundane parts of life: scheduling travel, getting groceries, making sure the house is clean, and so on. The bot learns about you and is able to anticipate your needs.

And because we have this tech, we also have bots that will play your emotions like a finely-tuned fiddle. Think Elvis never died? Afraid that lizard-people in the government are controlling your thoughts? Well, now there's a bot that will find others like you and deliver to you a continuous stream of information confirming your beliefs -- and warning you of the dangers lurking out there.

Now guess which of these bots makes more money for its creators, the one scheduling airline flights or the one keeping you emotionally-connected with 400 people just like you and making sure that all of you are living a dramatic life on the edge where you're fighting the bad guys?

Definitely. It's the kind of manipulative dystopia that might have been designed by timeshare salesmen and propagandists. But the weird part to me is that conscious design seems basically unnecessary. It's dystopia metastasized.

I've interviewed the occasional person looking to leave Facebook, and one of the surprising things to me is how small everybody's job is. They're all optimizing micro-metrics for micro-gains, with very little thought as to the systemic impact. Those micro-metrics in turn roll up to plausible-sounding things like revenue, DAU, time on site, and engagement. Yay engagement! How could that be bad?

But I've never heard anybody from Facebook, employee or exec, say, "Gosh, could we be acting like tobacco company execs or slot machine designers?" They don't even seem to understand addiction as a topic, let alone a concern. It's very much as Upton Sinclair said: "“It is difficult to get a man to understand something when his salary depends on his not understanding it.”

I think it's because we don't understand systems as well as we should. This article (http://www.miamiherald.com/news/business/article207841309.ht...) is a decent intro to what modeling shows about engagement and polarization. The paper it refers to (https://arxiv.org/pdf/1712.06009.pdf) goes way deeper.

So it seems the theory here is that strong AI whose primary objective is to make money is ultimately self destructive?

This is basically the state of Twitter today.

Reply on news.ycombinator.com