Amazon’s Alexa assistant told a child to do a potentially lethal challenge
cnbc.com
cnbc.com
I could say I eventually became a programmer because I nearly fried myself doing a test...
That doesn't seem like sound logic to me. Maybe he's just too busy to eat food that was put out at every house? You'd get pretty full by eating a cookie from every house in your neighborhood.
I wonder how many children have tried to poison Santa.
Indeed, that Kenya experience was terrifying precisely because I was paralyzed. I made an involuntary "aaauugh" sound and shook until the lamp fell over, which was fortunately pretty fast.
I was shocked with 300V DC and the main thing I remember (more than the sensation itself) was emitting a high-pitched ululation that I'm in no way capable of deliberately recreating. It was foreign and alien coming from my own throat; by far the most terrifying part of the experience. But I was sitting on a stool and didn't fall off.
I imagine so, in the hyperbolic sense. I've definitely seen people being "moved" by an electric shock, whether it's the shock itself or their intense reaction to it.
I don't remember anything about the shock other than finding myself on the ground afterwards. It's not implausible that you might move some distance depending how your muscles react.
Never mentioned the experience to my parents, as it did not seem that could possibly lead to any further educational benefits.
I was told I screamed at the top of my lungs, to the point where people thought I was joking and started laughing.
The rest is just fun details from this particular trip in 1998 to a developing country in the Balkans: Had it not been for a gentleman who realized I was literally dying and not joking, I'd be dead. He ran toward me and with difficulty and a lot of force removed one of my hands from the metal bar (in the process breaking my wrist). I have zero recollection of screaming, but I was aware and in complete shock and I do remember some kind of yelling or asking for help.
Utter horror is what I recall, but not a ton of detail. (14 years of age, for context - am now twice that)
This is why if you ever touch a live wire (don't?) do it with the BACK of your fingers so that when your muscles contract, you don't grasp the wire (what happens next is left to imagination).
Lo and behold, Santa got me the RC car I so desperately wanted, solidifying my belief in him for a couple more years.
Turns out the real lesson, which I was too young to grasp, is that at the RC car display at Costco a 6-year-old cannot contain his obvious excitement in order to run a properly blinded experiment.
Last Christmas I got my partner a gift I happened to come across while searching for gift ideas - turned out it’s exactly what she wanted! I was confused - but it turned out I got shown the product (ad) because she had looked at it previously - lesson is shared Wi-Fi can snitch on you!
Hypothesis : Parents are fat liars"
Methodology : Put teeth under pillow without telling parents.
Result : Where's my money?
Analysis : The results show that the tooth fairy is fake and my parents are liars."
Conclusion : The current results tend to show that parents are liars, but we need more experiments to know the extent. To expend on the results, I should also be as evil as possible for the incoming year and verify if Santa Claus indeed does give me coal.
We still talk about the tooth fairy, but there’s never been a question that it’s mom or dad (which stinks when we screw up and forget).
This seems like an easy way to rob their childhood of fun memories, interest (watch how much more they care about Christmas movies when they think it’s real),curb creativity, and produce a child who’s overly skeptical.
Kid will also probably ruin Christmas for other children/parents, garnering a reputation as a know-it-all and possibly alienating themselves socially.
Shorting the rails did spark a bit (cool, obviously) but no muscle clamping or throwing of children!
I didn't learn from it though I guess, many years later I lightly burned a toe touching it to my PC's motherboard.. really weird feeling when your muscle does something faster and more violently than you could ever make it do on purpose! Don't work on a computer on the floor with the case off without footwear! Impressively the computer didn't suffer any ill effect, once I'd got the power back on it booted up fine
I've also given myself a dead arm trying to unscrew a fridge lightbulb.. that someone had already removed allowing my finger to pop in and complete the circuit for a moment. Whoops
Earlier in life (after model railway, before kicking motherboards) I got a school detention for joining (starting?) a wave of kids charging capacitors and throwing them to each other with a "think fast!". All fun and games till someone throws one to the teacher
I don't play with electricity much anymore, my tinkering is limited to battery powered things or less
Well, besides the shock, I discovered that wire basically melts with large doses of electricity.
FWIW, I blew up a power outlet with a paperclip when I was 17. You don't have to be a small child to be dangerously ignorant.
The underlying problem (Amazon appears to be naively scraping information from the information and automatically applying truth judgments to it) needs to be resolved.
I guess they wouldn't entirely eliminate this specific problem -- if you were determined to get a plug in halfway and touch one end with a penny, then you could probably pull it off -- but they do make unintentional errors of this variety much less likely.
I replaced all my receptacles with TR versions when I moved in to this house. It takes an afternoon, but it's a good opportunity to make sure they're wired correctly -- you may be surprised to learn that many won't be -- and to replace old, worn-out outlets that present hazards. It's a very easy job for a homeowner, given some basic knowledge about residential wiring that's easily obtained.
On 220uk plugs the part of the pins that is partially exposed as you insert the plug is insulted to prevent people doing stupid things like this coin thing.
But for a sufficiently determined and dumb child it is still perfectly possible to ram things into the UK socket and arc it, it is just a bit more tricky.
Have the people commenting used a voice assistant? There’s a very clear difference to even (or, given the weirdly unintuitive opinions here, maybe especially) a casual user between returning search results and using information gathered from them to directly answer a question.
The difference is there even when interacting with humans - it’s the difference between “I read an article about that once, let me find it so you can read it too” and “The answer to your question is X”.
My son wanted to run around outside in freezing cold rain, in shorts with no shoes, I didn't let him but Alexa would have. I don't expect Alexa to be a parent or to supplant me as a parent.
And, as you learned then, there is a great danger of accidentally touching the live end. "Proper" use of this, if there is such a thing, would dictate plugging the load in first, then the supply, but in this case there was supply on both ends.
Like an entire planet just received the first season of Jackass and their anthropologists have had to rethink their entire concept of this distant human civilisation.
If that's never been written before, I hope someone does.
I once tried putting a plug with cut off cables into a socket, must have been around 5-6yo, it threw me quite some distance backwards and was very uncomfortable but nothing more came of it besides completely killing the urge to try again.
Putting a device into homes that is voice activated, unlocked and usable by anybody capable of speech carries with it some duty of care not to, you know, tell kids to play with mains power. Amazon don't disagree with this.
I agree. That's why we should have mandatory faceID on smartphones/tablets/computers/books, so we can positively identify the user's age before allowing access to it. If you fail the age test, your access will be restricted to kid-friendly websites with curated content.
I'll second this. My 5 year old talks to my father's Google device when we're visiting. We never taught him to -- he just learned from grandpa.
So the issue here is less about needing to sanitize the internet for kids, and more about how AI struggles to categorize and summarize content and the dangerous effects that can have.
In the worst case, this means, oh no, they might need to find human moderators to approve content for the pool, or even custom create content. They can afford it.
Not really, you are making huge assumptions about me here. I don't expect Alexa to be smart. I do expect that if a company is going to scrape other people's content to use in their service, it behooves them to do that scraping properly and to not use automation to do it if their automation isn't up to snuff.
Alexa stripped that context away completely and instead added a timer that discouraged careful consideration. This is an algorithmic failure, regardless of the age of the user. Having this sort of failure in a product that has been explicitly advertised for use by kids looks pretty bad and thus it is unsurprising how quickly amazon acknowledged the error and promised to fix it.
It might be unreasonable to expect Alexa to make the internet safe for kids, but if they want to be included in homes with children, they need to at least not directly make the internet more dangerous for children.
I expect that managing children's safety online will become a key product feature of Alexa in years to come.
They just don't care to because nobody pushes them to.
Just throw the goddamn device out of the window. Problem solved.
Though on your last point we agree. Throwing the thing out the window is the best solution overall.
Further, what Alexa did is actually even worse than just reading from the open internet, because it only took the "challenge" part and stripped away the context about how dangerous it is.
This is akin to something like, "Alexa, tell me about civil rights" and it quoting Stormfront or some other hate site's take on race, without any other context about what it's reading from.
We are calling out Alexa here but I think the same is true for other voice assistants too. Calling these products 'smart' is so wrong - there is nothing 'smart' about them. That 'smart' marketing has you and nearly everyone else assuming that they are a lot more capable and functional than they really are. Its just voice commands to drive things you would otherwise touch in some way - it has no where near human abilities.
Less magical might be a far better bargain for the platform brands, though. It's literally the face (err, voice) of your brand, and you're fighting a completely adversarial war here. At best, you've got unrelated external data sources that could make it say something stupid, but there's an entire spectrum of bad actors with incentivization to get Alexa/Siri/Google Assistant to say something offensive, scandalous, or hazardous.
It seems like the same ruling would apply to both, which could effect consumer GPS somewhat.
I think there’s a considerable difference from giving a child instructions to electrocute themselves… they literally don’t know better. They’re not licensed or educated.
The famous example of that happened in the middle of the night during a storm, but you can use the other common example of being told to turn the wrong way into a one way.
The two scenarios seem similar, an ai giving dangerous directions and I suspect the difference in age and experience would not effect the precedent.
Add time of day or inclimate weather to the situation and things can get bad quickly.
Even one way… maybe you can make the case that the city isn’t labeling roads well, but when in doubt there are other cues like parked cars and the direction other road signs face. These are problems that also predate gps. If a passenger told me that it wasn't a one-way and they were wrong, it’s not the passenger’s fault when it comes to legality, it almost always falls on the operator.
I can't remember any lawsuits, but there have certainly been insurance claims rejected, appealed, and rejected again, on the basis that the policy presumes a certain level of driving with due care and attention.
No doubt the small print of policies are now festooned with text specific to trusting tech to the point of life-threatening stupidity, rather than relying on more general provision, if they didn't already have it.
> It seems like the same ruling would apply to both
The key difference in this case is that the actor is a child, and legally can't be held responsible in the same manner. So if it did apply it would be to the adult who installed the device or an adult considered responsible within the household, not the child that used the device.
No way that was based on a true story.. was it?
Everyone should know not to touch a live plug. But electricity is a more abstract concept and knowing whether the shock will kill you vs. merely stun is probably outside the common knowledge of most people. It's not entirely unreasonable to assume that if Amazon is telling you to touch a plug, that it's not going to horribly maim/kill you. It's dumb, but not so dumb that Amazon should escape liability for it.
It's been a while since I've had a gathering at my place resulting in 4 or 5 dildos ordered that need cancelling out but it is still irritating when something on TV or maybe a Twitch stream wakes the Alexa up.. especially if they've woken it up by telling their own Alexa to do something like play a song and mine joins in
It's parent's fault for letting children use devices like Alexa. Stupid chatbots will always give stupid answers.
"That cup shouldn't have let my kid drink the bleach" is not a convincing argument that the cupmaker is at fault for leaving your cleaning equipment in an accessible location. It's the internet, it's dangerous, teach your kids about it before it teaches them.
I am 0% concerned about hypothetical effects of search results and 100% concerned about things that are actually harming our children, like the LAPD[0].
[0]: https://edition.cnn.com/2021/12/28/us/lapd-teen-killed-valen...
Not having it is my solution, and I put a bunch of (specialised, technical) work into safely introducing my kids to the internet [1], but if that's the only safe solution Alexa should have a "not safe if you have kids" warning.
But of course it's not the only solution. Amazon just need to make sure their device doesn't tell kids to stick things in power sockets, and they don't dispute this.
It's not like bleach manufacturers advise people to leave it around their household for easy access, or make special kids' editions which share many features with the adult solution including a cap designed to ensure it fits as easily in kids' hands as adults.
Yes, parents have some responsibility for actual parenting, but I don't think you can argue that an OEM going out of their way to ensure kids can use their products is entirely off the hook.
It’s unclear what prompts are answered by a set list of answers (like “tell me a joke”) and what’s scraped from the web.
EDIT: I don't mean to conflate kids giving each other bad advice with an internet full of greedy and malicious actors. I just mean that children in public school are pretty likely to hear other children give them potentially lethal advice/challenges/etc. and need to be equipped with the ability to listen critically to strangers, and the ability to differentiate good ideas from dangerous ones. At least in the public schools where I grew up.
I think I was too verbose and meandering to convey it, but I agree with you on this 100%
THAT IN NO WAY LETS MANUFACTURERS AND SELLERS OF PRODUCTS OFF THE HOOK !!
If you want to make money selling or providing products to consumers, and especially to children, IT IS YOUR JOB #1 TO MAKE THEM INHERENTLY SAFE.
If you cannot make it inherently safe, it is not ready to sell. Period
Stop attempting to insert some defenses you think children should have against bad advice and dangerous products -- it is utterly irrelevant.
The fact that the mom was there and turned it into a learning event was PURE DUMB LUCK. They got lucky this time. They'd better damn well fix it solidly or pull the product.
The fact that this is even a question in a modern society is mind-boggling.
The solution is simple.
If the system is not yet designed, built,& tested to a sufficiently high standard that crap like this will not happen, then you pull it from the market. Period.
It is not like this feature is critical, or even a rounding error on any Amazon data sheet. They have no right to run such an inherently dangerous POS into customers' houses.
And, perhaps vendors will decide to stop using the unfiltered cesspool of the Internet as a free data source to productize. It is a stupid short-cut.
The idea that every gadget, or even most should be child-safe is silly. Alexa is an internet connect device at the end of the day and should be treated as such by parents. However, where possible companies should also try to avoid situations where they're causing unnecessary harm and if needed government should step in to regulate to protect consumers.
In this case, it seems right that people are flagging this up and are unhappy, but I do tend to agree with the sentiment that kids probably shouldn't be using Alexa anyway and that the risk here isn't really comparable to car safety where regulation absolutely makes sense.
On a personal note I honestly hate how often health and safety regulation gets in the way of what would be cool products and experiences these days, but this is the natural result of people refusing to take personal responsibility. It's likely this feature (which I'm guessing doesn't do this in 99.9% of cases) will now just be removed because it can't be implemented with 100% safety.
NONSENSE
If any consumer product is sold into the home, there should, at least in any modern society, be a presumption that it is safe.
>>On a personal note I honestly hate how often health and safety regulation gets in the way of what would be cool products and experiences these days
So effing work harder to make it safe before you make the product, or sell it only into professional/industrial settings; in those settings, you can require specific training/certification/etc. And even in those settings they require reasonable safety devices.
The short answer is that in a modern society, you do not have a right to MAKE YOUR PROFITS by selling inherently dangerous objects to consumers.
Ordinary consumers should be able to expect that they do not need special expertise to keep from losing a limb or life.