Time to regulate AI that interprets human emotions?

Yeah my limited understanding seems to indicate that what I'll simplify and call 'careful / consistent data formatting' is required before AI can really do much with it.

Humans pounding on the keyboard (or even just selecting from a list) is probably painfully not 'careful / consistent data formatting' and thus a huge obstacle.

But the result of being disconnected and no real feedback is the result none the less.

My limited understanding seems to indicate that for AI to process input it has to be handled / processed carefully for any good to come from it.

Accordingly making decisions based on human feedback could be super difficult / impossible at the moment.

The result though seems like a super narrow minded system that wouldn't know when it is dead wrong ...

Anecdote time - I use Apple music and am generally happy with it. However, some time ago I started putting on the 'Sleep Sounds' playlist before going to bed. That's messed up my recommendations for my 'New Music Mix,' a more valuable feature for me. Even after disliking the 'Sleep Sounds' and specifying that it shouldn't recommend more music like it, my recommendations are still messed up. Makes one wonder to what extent in the age of 'AI' whether everyone is an edge-case in some form, particularly when (as you point out) context is a big mess.

Competitors, people actually wanting you to buy said product, the business selling since they get a cut, companies that process the transaction since they also make money. (no idea on regulations)

Why didn't you buy said product outright and instead put it on a wishlist? Lack of money? Price too high? Dubious quality?

Most things on my wish list remain there un-purchased ... forever.

But they'd rather advertise random products to me?

Why not instead just advertise related products to what is on my wish list?

I have to push back on this subtle anti-humanism. No other animal species would even hesitate to make use of any and all advantages. That’s called evolution.

Humans, at the very least, try to stop and think before doing things.

I've seen a lot of Big Tech engineers make this same quip to avoid having to take any responsibility for the horrible results of their maliciously designed systems.

>Welcome to humans. They suck.

If AI is being developed by humans who have anti-human beliefs like this, we are in trouble.

People with emotional disorders either do not define boundaries or do not define them effectively, and they often do not respect others boundaries.

You could use this analogy to imagine an ad agency as an emotionally disordered person, and you wouldn't be too far off the mark.

Advertisers DO NOT GIVE A CRAP if you are offended, or annoyed, or troubled. All they care about is if they send out 10,000 ads, they get enough increased sales to offset the costs of the ads.

That's the fundamental algorithm behind all the fancy AI technology.

That’s a good start, but it barely scratches the surface when it comes to the dangers of AI emotion recognition.

Also, those Muse headbands that read your alpha brainwaves which are used for meditation are dangerous. While they are incapable via (hardware) technical specifications of being able to determine your emotions via code, it can determine whether your mind is in a clear or distracted state. Hence it is used for “meditation” and is considered to be “beneficial” for that purpose.

But, of course it nearly certainly can be used for perpetual harassment, especially since third-party apps can interface with it. Is your mind clear (according to the headset, via code)? Alright, time to bombard you with harassing adtech. Also, it benefits the “designers” of these apps as it makes you more dependent on the device.

The same goes for other consumer EEG devices. The ones that are used with sleep like the Dreem 2 can be used to make your life a total hell through unethical experimentation and more.

I honestly have no hope on such issues. Spotify AB (a Swedish based company), for example, is believed to violate the GDPR. Not only that, they have a patent pending for emotion detection based on your song listening/streaming pattern. There are projects doing this for Spotify doing this exact thing publicly available online at GitHub of course.

you'd be better off thinking of who benefits and what their incentives are. Because people by and large are acting based on incentives, and if you can reshape the incentive structures then you can discourage the action (by opposition eg. making it punishable) or remove the incentive altogether (through systems interventions).

Blaming it on politics leaves you with no actions to take beyond trying to talk people into acting against their incentives - which you can do, but it's not easy to get people to change their minds en-masse.

I would like to push back against this a bit. You, along with most people in this thread, probably suffer from availability bias. People only interact with AI/ML systems when they are surfaced to them, and currently the most popular channel to surface ML is ads. Of course I have my own biases, but I think is an important critique. As Sinclair said, "It is difficult to get a man to understand something, when his salary depends on his not understanding it."

> It's not profitable to use AI for wholesome and healthy things for people, so nobody does

The purpose of the ML systems I build is to detect when things go wrong in order to keep people safe. These systems are both important and extremely valuable. They just aren't quite as sexy as the latest computer vision or large language models. But that's ok. Normal people only become aware of such systems when things go horribly wrong. It's sort of like plumbing in that regard.

Every post on HN with this question receives the same response: “it’s because Amazon knows you’re actually more likely to buy another one if you’ve just bought one, even if it’s the kind of thing you only need one of (e.g vacuum)”

I find that answer unconvincing and there is no published data to back it up, but it is repeated constantly here.

Caveat: I don't work on advertising systems, but I do work in ML, and I've studied recommendation systems. One way to think about these large systems is that they contain a condensed representation of you that is comprised of all the things you buy/consume on their platform (the raw data) and some derived descriptors (possibly demographic information/information from other platforms/"engineered features" like what "style" of clothing you like, etc. This representation of you (a vector) can be compared to other members of the platform (inner product), and they can give you recommendations based on your similarity.

The problem is that in this case, they've encoded into your condensed representation something about office chairs. A better system would be knowledgeable about the kind of purchase and infer if it's the kind of thing you would buy once, or multiple times.

But then you have the problem of different individuals having the propensity to buy different products at varying intervals. If I like shoes, I might buy many pairs. You might not be into shoes, and so you're not going to buy more than a couple pairs, if that. In this case, how do we train a system to know the difference between the two of us, with our different tastes? The more specific we get with the vector descriptor, the harder it is to compare us to other people (because in high dimension, random vectors are basically orthogonal).

And going through all products and trying to determine if they are the kind of thing that you would buy more or less of, given that you just bought it, is expensive, and likely difficult. The system ultimately falls on a positive correlation between history and future purchases.

Another way to think about this is with Bayesian reasoning, where your prior likelihood to buy office chairs affects the posterior. The system's prior is high (you've bought chairs in the past), which scales the posterior (that you'll buy a chair now).

It might be simply driven by statistics, or as a human-directed strategy.

While it seems counter-intuitive to the person who is targeted; (examples abound) - if you think about it from the perspective of someone selling widgets, the (set of people who are in the market for widgets) will necessarily include people who just bought one. Maybe they are not satisfied with their purchase? Maybe they want to buy another? Maybe they bought one, and it arrived damaged?

Personally, I would think that this kind of "trying to think FOR me" is rather offensive to a lot of people, and probably turns-off a fair amount of people who are subject to it. On the other hand, it is probably true that people exposed to an ad for a widget, immediately after buying one, are statistically more likely to buy another. So as long as that is true (if it is) - then that's going to be a criteria for ad targeting.

That means the recommendation system isn't a neural net trained on the buying habits of users. It's a simple logistics system that recommends items similar to the ones you've bought.

Amazon presumably doesn't share purchase history with advertisers; you get lumped into a cohort with certain interests/keywords.

An experiment to try; if you only go shopping/searching for vacuum cleaners but don't actually buy one you are likely to see the same phenomenon.

Why do advertisers pay for this? Because conversion statistics show that showing ads to people shopping for vacuum cleaners do actually result in vacuum cleaner sales. A/B tests back up the causality of the results, so vacuum sellers pay for the ads.

Why doesn't Amazon remove you from the cohort of vacuum cleaner shoppers after you make a purchase? 1) it would reveal that you made a purchase. 2) Amazon would lose out on additional ad revenue.

I have no inside information on Amazon but it matches what I've seen on other large ad networks.

I've been puzzled by the same question (as a reaction to the same ads) and it occurred to me that it may be simple math:

If the likelihood of buying by someone who already bought the item is 3% and the likelihood of buying by someone else is 1% (simply because there are many more people in the second category), which group would you focus on with your ads?

Personally, if I like an item, I'll recommend (and gift) it to my family and friends. I can think of at least 3 items that I bought for myself, liked and bought for someone else that are normally more of a long-serving items (like a tea kettle or a vacuum).

I can't speak to amazon specifically, but my understanding of this phenomena in general, especially with ads that are bid for, is that the systems which track and gauge your likelihood to purchase aren't connected to the systems that know you did make a purchase, at least at the point in the system where it's making the decision of what ad to show. It just sees that you've done things that a person who is likely to purchase that thing has done, which, of course you do, because you DID purchase that thing. It's like the ad is chosen based on the strongest additive signal available, and negative signal isn't considered.

The likelihood is that the chair you got, you'll probably find out it is just bad and after a while you'll return it and look for something else. In the last few months almost 50% of my purchases turned out to be some junk pretending to be a quality product. I still have a few boxes of things that I need to post. There is even no point reading reviews, because most of them are fake. I think I am going to stick to trusted brands and probably drop Amazon altogether, but I like the next or same day delivery a lot. When using other stores you never know if the product arrives tomorrow, next week or never.

I'd believe your proof if somehow there was something to compare it to....

In the meantime I get ads for stuff I'm not buying, so I'm not sure anyone knows what 'reliable' is.

Technicality people do not always express emotions the same way physically despite what may be the "consensus" on what an expression is believed to be. It can differ culturally for one, let alone neurological conditions or neurodiversity. Human vision isn't strongly reliable - just the current standard. If somebody claims they don't have HIV when their tests come back positive it is clear which one wins out.

Well, that's actually relatively easy if you know what you're looking for -- real smiles are known as "Duchenne smiles" and involve contraction of eye muscles. And that could absolutely be done by AI today, relatively easily.

The issue is the emotion behind it... you can be quite happy without smiling at all, and people who pose for the camera all day long learn to give a 100% convincing Duchenne smile no matter how they're feeling that day.

First, you need reliable training data to begin with. This can't be supplied by third-party labeling of faces, so it would need to be done with a dataset of self-reported emotions, but self-reported emotions also have tons of pitfalls well documented in the literature.

But second, the correlations of individual facial muscle contractions with emotions has been extensively studied and it's far noisier, inconsistent, or completely devoid of a signal than many people assume. In academic terms there's no such thing as a reliable emotional "signature" to be gleaned from facial muscle activation.

So the point is, it appears that the raw data simply isn't there for the AI to detect patterns that humans can't. Detecting emotions requires far more data points outside of facial muscle activation -- such as the ones I listed.

This is a common misperception about AI and ML. Most leading voices in ML will tell you that--unless you have a very good reason to believe differently--ML is not the tool to turn to if you need better than human accuracy. That also comports with my experience in the field over the last 6 years.

> This is just setting the bar far too high... It's far better than random chance

I don't think it is setting the bar too high, "far better than random" can still be abysmal.

You're right that humans are better than random, but there is a wide range of emotions. Assume for the sake of argument there are 10 emotions to identify and evenly distributed in frequency, then random is getting it right 10% of the time. Let's say humans get it right 30% of the time from still photographs of real-life situations (not staged) -- I'm making this specific number up but depending on the type of experiment, it's a reasonable ballpark one.

That's still horribly low, wrong more than half the time. I can't think of any situation where that could warrant any kind of responsible decision-making. If we were talking 95% accuracy then there could be value, I'm not asking for 99.99% accuracy. But if reliable detection is something like 30%, I stand by calling this "snake oil" even if it's not just random.

Also, it's conceivable for a machine learning algorithm to surpass humans in certain aspects, e.g.

- Identifying emotions of humans who grew up in a different culture, for which the ML algorithm may have piles of data from every part of the world and you don't

- Identifying emotions of babies based on instinctual behavior

- Identifying emotions of people who are undergoing a trauma response that most humans are not trained to recognize

- Identifying emotions of people during a negotiation process in which the people are actively trying to hide their emotions from humans but nevertheless leak certain signs of their true emotions

I'm well aware of the current state of AI, thanks. You, however, may need to familiarize yourself with the current state of emotions research.

Sentiment detection in text, for example, is not emotion detection. That's not a rebuttal.

As an example of what I mean, you can feed a NYT news article written by a journalist experiencing regret, or boredom, or elation, due to events at home, while writing it -- and it won't pick up a thing. But it may pick up a valid sentiment of outrage, which was intended by the author as the desired effect upon the reader, but which was not once felt as an emotion by the author while writing it.

Semantic detection is far easier because the author of text generally tries to communicate a sentiment, consciously, in their text, and text is intended to have fairly unambiguous meaning. Neither is generally true of emotions on the face.

And nowhere did I say emotions only exist in static images. Of course they are communicated through voice, video, etc. -- I don't know how anyone could think otherwise.

> about Zooom videos, not single frames

The products in question are almost certainly assessing still frames from video. There's been vanishingly little research on the time component of emotions, even though we know it's hugely important.

> you need it to work cross-culturally

This would then require separate training models for e.g. individual subcultures within countries as well as detecting which subcultures participants belong to. That is also far beyond anything being done currently.

> but many settings are not adversarial, so there would be no reason to do so

It has nothing to do with an adversarial setting, people try to hide their emotions constantly. They hide that they're fed up with their boss in front of colleagues, they hide that they're stressed with their spouse at work, they hide that they're worried the project will fail. We are emotionally regulating virtually all the time.

> some emotional information does get communicated over the video, so in principle computer programs should be able to pick up on it too

In principle, yes, but in practice the emotional content is so dependent upon your cultural and individual mental model of the person that you would need to model their entire psychology. "What does that long pause mean?" The point is that emotional signals are so incredibly complex and vary so much from person to person, that the difficulty of accurately decoding emotions is more akin to AI that can make conceptual inferences and hold a genuinely intelligent conversation, as opposed to mere pattern recognition.

It's at best sloppy to just say "this needs to be regulated, but I can't be bothered to suggest even one possible regulation that would fix anything I find so objectionable." The author has a doctorate and works at Microsoft Research on AI. If she thinks it should be regulated (which is absolutely a fair position to take) I'd much rather have her be the one suggesting regulations than some 80 year old lawyer who's been in Congress since the civil rights era or some special interest lobbyist.

> those who wish to undermine your ideas can attack your implementation rather than the concept.

Yeah, it's frustrating when people do this. But if you want to have real solution at the end of the day, then you'll need to hammer out all of your implementation issues.

If you produce an implementation that is flawed, then people will be able to evade the spirit of your regulation rendering it useless.

I find this to be an interesting statement. What happens if you're able to evade regulation by making your "AI that interprets human emotions" of really poor quality.

"Your honor, we investigated the source code and discovered that there was technically no AI being utilized. It was a poorly formatted series of if-statements. Honestly, I wouldn't even consider this a program. How it avoids crashing the moment it's run is beyond me."

Like, does AI mean some statistical method is used? That large data sets were used? Are you using some declarative language like prolog?

Ultimately the only definition I can see them coming to will reduce to "person uses computer to do thing I don't like." And somehow I don't think that's going to actually help.

People can read other people's emotions. You can build a mechanical turk program that's effectively the same as having a personal concierge agent with respect to the regulation.

Like, if you ban reading people's emotions you effectively have to also ban any human interaction.

Suppose there's a browser plugin that uses sentiment analysis on text fields to try to detect text written in anger. And it doesn't do anything with the information except warn you so you can think twice before posting an angry comment to social media or sending an angry email. Should that be illegal?

Or maybe it more directly tries to sense emotion by using your webcam to look at your facial expression.

Suppose someone, for example a depressed person, wants to use it to monitor their emotional state and use various automated responses like notifications or distractions to help maintain an emotional state they want?

I think this viewpoint is limited. Sometimes reading peoples' emotions can be an extremely good thing.

I am aware of a healthcare company which currently has a model in production which alerts healthcare providers if a person displays suicidal intent. They have several confirmed instances deaths being prevented because of the interventions taken due to alerts from their model. While I don't think ML should be used to manipulate people's emotional states, I think this is a case where having a model that can read people's emotions is a good thing.

The funny thing is that by totally banning "anything that purports to read people's emotions" you also ban research that would help people. For example you need to train models to detect race in order to make sure there are no racial biases in another model you want to deploy.

Personally, I think this type of thing falls into what I like to call the "all problems are management problems" category.

If a TSA agent deports someone simply because the machine recommended it, then that is a problem. But if a TSA agent deports someone simply because they were having a bad day, then that is also a problem.

I guess I don't care what tools those in authority are using (their intuition or a mechanical intuition or a database lookup), but what I care about is whether or not innocent people without the ability to navigate an appeal system are being erroneously penalized.

And ultimately, it's the responsibility of management to make sure that the people/tools they assign are doing a good job. If they fail, then management needs to find new people/tools. If management fails to do that, then all the AI regulation in the world isn't going to do much good.

> decision making ... should not be automated by machines. There must be a mechanism in place which is open to scrutiny (preferably by a qualified human) and accountability

You're saying "I don't trust AI, I want human supervision", but this works both ways. Sometimes we don't trust the humans and would prefer a neutral AI. Humans do terrible things to other humans. Who's going to review my appeals? What are their biases? Are they any more trustworthy than a model?

That's the problem. Those who most want to do that job are the ones who will abuse it the most.

I don’t know if there is anyone that knows for sure if it is possible to create an AI sophisticated enough to detect skilled professional acting.

Though, if we allow our imaginations to take off a bit, one could imagine an AI that can represent the entire mental state of an individual in itself and probe it to determine if you’re acting while you’re acting in front of the AI.

> Conmen and various methods of crime require significant acting ability

I really don't think they do. The main thing that they require is that you don't listen to the anxiety (or after experience lose the anxiety completely) that everyone can see through you. It's the illusion of transparency that outs a liar, not any codable authenticity in their reaction. "Tells" are nervous tics, and "lie detectors" are sympathetic nervous system arousal detectors.

People can't see through you. A funny thing you learn when doing public speaking is that they can't even see your crippling nervousness unless it expresses itself in stereotypical tics. If your tics are strange, people will have no insight into them (unless they know you.)

https://en.wikipedia.org/wiki/Illusion_of_transparency

They agree. It's just that a) sometimes people lie about their intent; and, b) sometimes the courts have determined intent is not the most important thing, or even irrelevant.

The courts can discover intent. It cannot determine it independently.

ADS are not the worry I am expressing here.

Other things, like tests to judge the nature or inclinations of someone are a much higher worry.

You do not even have to assume it - here's a quote that sums up the drift of the entire article: "It is time for national regulatory agencies to guard against unproven applications, especially those targeting children and other vulnerable populations."

The article also makes the claim that some of these technologies are based on simplistic and largely disproved psychological assumptions, but again, its opposition is to their use rather than mere existence.

Because one is a piece of data that anybody can create and distribute, while the other is a harmful practice that people are fighting for generations.

If x is true, you have no right to prevent anyone else concluding it about you. Truth is the ultimate defense against all this anti-science FUD.

No, but human brains weren't engineered by humans, and so it is reasonable that we don't currently understand them even as reverse-engineering efforts continue.

AI, by contrast, is engineered by humans, so why is it humans can't explain it?

This is not an issue of concentrated private wealth. Most schools are run by governments.

You are right, citizens need to be empowered against their governments. But regulations give governments more power, not less.

And if you say the majority of consumers doesn't care about some issue, what makes you think democracy could provide a solution?

You also don't need that many feet to create an alternative school. You just need enough children to justify an extra building and teachers, or supplement the home schooling.

have you considered that perhaps companies and governments are together on this?ok, governments do slow down things and come up with regulations here and there (if they decide to). Imo, at the end of the day, they both play on the same team...

I'm also an European, and I disagree strongly. Not everybody in Europe is a socialist.

The only thing that guarantees fair treatment is competition (so that for example a worker can quit and work somewhere else). Government rules just breed corruption and inefficiency that harms everybody.

As an industry of computer SCIENTISTS, we should make the statement that such systems are not possible and fraud.

But your mind cannot identify the emotional state of a professional actor. People are far more sophisticated than this immature and naively conceived technology.