Comment by mwfunk | Hacker News Reader

Comment by mwfunk | Hacker News Reader

He's addressing one aspect of AI that can be disturbing to some people, specifically becoming reliant on the expertise of others in your day-to-day life and thus becoming less independent. If your biggest issue with the concept of AI assistants is that they might have insights about you that you yourself don't have, I could see this article making someone feel better about it.

For me at least, that's not the primary fear. I don't fear people becoming dependent on AI. Rather, I fear people misusing the information that AI reveals about others. I don't feel uncomfortable from (for example) AI observing my media habits and using that information to make recommendations for other media.

However, I would feel uncomfortable if someone with access to that AI then fed those insights into some other AI that I wasn't aware of for much more nefarious purposes, such as profiling me to try to quantify my loyalty to the government.

That's just one example. Even if I had absolute faith in my government, bad actors could misuse that information to do other things, like determine which people would be most likely to take on debt on unfavorable terms (so they could be sent a 20% APR preapproved credit card, naturally!), or which people would be most likely to want to help a troubled Nigerian prince who just needs a place to temporarily park a bunch of money.

I'm not trying to fearmonger and I'm actually very excited for the future of AI. I just don't think this piece is addressing the deeper fears people have about the technology.

The problem with habit reinforcement is that you've simply created a feedback loop.

"You like [music type]? Here are more bands/artists of [music type]."

It looks like an innocent service. But it's devastating to real exploration, because it makes it much less likely you'll ever discover Band/Artist Z whom you'd never normally listen to but love anyway.

This is why thoughtless customer profiling is a dumb idea, and certainly not the sure fire insta-profit marketing panacea it's sometimes supposed to be.

It has some uses, but you're reducing customers to stimulus/response robots with a limited behavioural repertoire, and that's an excellent way to miss a lot of opportunities.

If you however do succeed in reducing humans to stimulus/response bots, it will likely be good business. Which is the scariest aspect, cause that drives lots of decisionmaking today...

> The problem with habit reinforcement is that you've simply created a feedback loop.

> "You like [music type]? Here are more bands/artists of [music type]."

So far I've found the opposite. Pandora has exposed me to music that I never would have sought out on my own. In fact I can think of several artists that I would have judged by their cover, so to speak, and never given a chance even if I had stumbled on them on my own, instead of coming to awareness mid-song that "I don't know what this is, but I kind of like it."

My wife teases me about this a little bit when she finds me listening to music that she says isn't 'me' or doesn't seem like something I'd like. She's right, but I guess Pandora knows my tastes better than she does (and better than I do, to be fair.)

Contrast that with Spotify, which when given a song or artist that I like goes out of its way to recommend music that is superficially similar but that I hate. Or how it can recommend 80 songs to me a week based on things I have listened to and end up recommending me nothing that I like.

Huh. I've actually found the "Discover Weekly" feature to be remarkably good at picking music that I like. I'd say I affirmatively like about half to three-quarters of the songs per week, with only a couple that I find myself skipping.

I wonder if it's better at certain genres than others.

> I wonder if it's better at certain genres than others.

The Discover Weekly algorithm is, as I understand it, based on what songs have been added to other user playlists. So if you're primarily listening to a genre that has a lot of intense people making carefully curated playlists, you're gonna get a better Discovery Weekly.

And more generally, I would say it just depends on the quality of the AI. There is no fundamental reason why such a system would have to recommend only similar music. It could try contrasting music, and build a picture of what aspects of the music you like. It could draw conclusions to recommend seemingly different genres which nonetheless share some commonality that might make them appeal to you. It could also occasionally try out completely different things, to broaden your horizons (and give you more of that if you react favourably). And that's just scratching the surface. You could do all that without AI. An AI could potentially do all that, plus other interesting things that we wouldn't even think to try.

The algorithms used to make musical recommendations do statistical analysis on numerical ratings given by different people to different songs. They typically know nothing about music, as musical knowledge is understood not to improve their results. I wouldn't call them AI - they're neither knowledge-based systems nor neural networks.

I don't think that that's true for Pandora: https://en.m.wikipedia.org/wiki/Music_Genome_Project

> I wouldn't call them AI - they're neither knowledge-based systems nor neural networks.

Out of curiosity, why are you drawing the AI line at neural networks? Neural networks are just function approximators; there's nothing magical about them except they work really well on problems we're interested in.

Would you really not consider an SVM or a Naive Bayes classifier "AI" since they don't use a very rough, very loose analogy to the brain?

I discussed this problem decades ago when I developed MORSE (http://web.onetel.com/~hibou/morse/MORSE.html).

People like certain things for no obvious reason, and are unlikely to change. It does them no favours to recommend them music they don't like, or anything else which depends on taste. But if there's music similar to the music they're known to like, which they haven't heard - perhaps because the band is obscure, from a different decade or part of the world, or new, for example - they might want to hear it. Also, there might be one or two songs they'd like by artists they don't normally listen to. Collaborative filtering is good for this.

Your point is valid about ideas, news, and a number of other things which have nothing to do with taste. People would benefit from sometimes being exposed to viewpoints different from their own.

Hence random mutations in genetic algorithms. And other similar functionalities

I still think your examples are more of the "powerful tools" kind of AI. And I think the general argument about "hard AI problems, when solved, cease to be AI" is also along those lines.

I think we've proven by example that what we thought of as things that require "real" intelligence, like chess, go, jeopardy - really don't. They're not a good yardstick for intelligence. Or intelligence is not as closely tied to (self) awareness and consciousness as we once thought.

I do worry about the "autonomous system" problem. The link between meta-data and drone strikes have brought us uncomfortably close to a real Skynet - without any need for an emergent consciousness per se.

But I always read discussions on AI (the kind that "Wake up") to really be more about AC: artificial consciousness.

> I don't fear people becoming dependent on AI.

Indeed. I fear AI that stops being dependent on people. Such systems are still far off, I believe. But I don't see why we couldn't intentionally or by accident create a new form of life that could out-compete us, and in the end make us extinct.

> Indeed. I fear AI that stops being dependent on people. Such systems are still far off, I believe. But I don't see why we couldn't intentionally or by accident create a new form of life that could out-compete us, and in the end make us extinct.

Since the line between AI realized in a machine and systems in general is kind of thin and blurry, one could argue that the economy is about to become such a system. If automation can fill all the links in the feedback loop chains of the market, we could one day wake up to see humanity being completely excluded from the economy, which would continue onwards by itself. A Disneyland with no children[0].

[0] - http://slatestarcodex.com/2014/07/30/meditations-on-moloch/

Absolutely. Just as we are (in my opinion) a result of emergent systems (evolution) - the systems we build should be expected to have emergent properties. And there are other systems we are beholden to (and impact) such as the biosphere and the climate.

Criticism of capitalism is often misunderstood, or misconstrued as "conspiracy theory" - while what it often is, is just pointing out the most likely and apparent emergent properties of capitalism - and likely follow-on effects of that. Examples are concentration of capital and power, ignoring "external costs" like environmental damage etc.

Personally I think "emergent" != "emergent". Most of the systems we build are pretty dumb - except our socioeconomic ones, which gain complexity by being formed by aggregating over billions of sentient beings.

Now I'm having some trouble sorting out the categories in my head. On the one hand, the "emergence" you described in your comment is garden variety feedback loops. Something very simple in principle - and being a part of a feedback system literally means 'being beholden and impacting it'.

On the other hand, it really feels like search for intelligence is in a big part just tweaking one's abstractions. You can see the market as "just" a complex set of feedback loops between individuals and groups, or hop one abstraction level above and realize that we're all components of a feedback system that's optimizing for something we as individuals have close to zero control over. A system that's bossing us all around. We try to understand it, so that we can say that e.g. prices of product X dropped because people in company Y invented some innovative process. But if one digs deeper one can sometimes notice that if it wasn't company Y then it would be a company Z; if it wasn't an innovative process then it would be an innovative business model (or a scam), or a substitute product X1. All because the "market pressures" made it happen. If you step another abstraction layer up, you can just treat the market as a being and it's not a big of a stretch.

So at what point one should stop playing with abstractions? Or, maybe, what do we mean by "intelligence"? If we're looking for a different-but-mostly-similar being to talk to, then searching for it by doing abstraction dance is not a good way. But if by "intelligence" one means a sufficiently smart optimizer, one that can make decisions to reach its own goals, and that in principle could coerce us to do its bidding - if one means that, then one may realize this has already happened.

Some of the things attributed to capitalism have nearly nothing to do with capitalism itself. I'm starting, for one, to think that there's probably less concentration of wealth and power than we think. Bill Gates is a heck of lot wealthier than was JP Morgan, but Morgan had a lot more concentrated power and wealth than Gates.

Not to quibble, I just think our analysis is deeply flawed.

Reply on news.ycombinator.com