Hacker News Reader

Comment by tantalor | Hacker News Reader

I'm surprised by the overwhelming positive vibes in the comments here.

Maybe I'm alone? To me, this comes across as extremely creepy, the exact opposite of what we should desire from AI in products aimed at children.

View on HN

Totally get the creepy part, but my criticism of devices like this is that they seem to be made by people with limited exposure to the creative power of children.

Children don’t need this; they are so much more creative than an AI (and the adults that trained the AI), and their creativity is fueled by boredom.

I mean when I was a kid I had action figures and played out scenarios. Would be pretty nuts if you could make your own TV shows with AIs assisting the play. Or set up your own battles, etc. Especially if it had more animatronic entry points

I feel like it would be creepy if the kid was using it without anybody ever checking up on it ... but I think all of my friends with kids would say that the answer to that is "parenting."

I mean, giving a kid an unlocked iPad and not bothering to do basic supervision can also have really creepy results, so I'm unconvinced that something like your work actually makes anything worse in the negligent parenting situation, and seems like it could be a lot of fun in the competent parenting one.

If you haven't already done this, I'd note that I can think of a number of parents who would probably rather enjoy a version of story mode that let them collaborate with their child and your code to put together a bedtime story before they turn it off for the night and tuck the kid into bed.

> with limited exposure to the creative power of children.

This is true, I am not a parent. But I have some domain expertise in building a conversational toy... talking to many parents and having been a child myself for several years has helped

You'd be surprised how un-creative many children are.

I doubt that. I have two kids, 4yo and 6yo. I told my kids that I can make their toys talk (using AI) a few months ago, and they have been constantly asking me when it will be ready.

I commented that I like the project, in that it is a project that helps you to create a realtime assistant - i would love to replace alexa/siri/whatever with something actually useful.

That said, I totally agree that I wouldn't want this in a kids toy. The whole idea is super creepy in that respect, with so much scope for abuse.

For parents we added a `Story mode` option (similar to Yoto toy / Toniebox). The idea is: the AI crafts a story and invites the child to craft the story together in a more engaging way. The story prompt keeps the story focused and in scope.

I hope these toys could be a joy/comfort for kids that don’t have a parent that cares.

I poured hours into games/programming because it was a happy place away from school etc… These toys could be the same.

This technology is neutral, but I see so much potential for projects that do good.

The Elato toy is currently not aimed at children. The current version has adult characters that are entertaining and fun to engage with like the Chad Brew Barkley character in the videos. I put up more such funny videos on my tiktok tiktok.com/@elatoai

However, while testing it with a friend who has a 5-year old daughter, I added a `Story mode` feature to create dynamic stories for her which she enjoys.

I think what would be even cooler is if each character in a story has unique voices (like voice of an ogre, voice of an elf etc.) which is currently unsupported in the single websocket connnection.

Why is the idea of a child talking to a LLM creepy? Do you think a child is gonna figure out how to jailbreak the “keep it keep kid, friendly” prompt, and start talking about I don’t even know what … kids don’t know about adult things. That’s just not how kids be.

I genuinely can't fathom how it wouldn't be creepy.

Bots are for doing tasks. I don't want to socialize with them and find the idea of kids being socialized by bots supremely weird. At least the AI girlfriend people are (probably unwell) adults.

> Why is the idea of a child talking to a LLM creepy?

The target audience is young kids who are still developing socialization skills. This toy off-boards that development from a human to an AI. We don't really know how that affects a kid.

This also plausibly trains the kid to think of other people as AIs: subservient tools that exist primarily to respond to them. Not exactly a healthy attitude to take towards one's peers.

It's presumably also going to get a lot of unsupervised usage, and the occasional AI model updates. What happens when a bad model update has it advising kids that soap is a forbidden candy that tastes delicious?

(I'm not saying any of these is particularly likely, just trying to share the sort of concerns that would lead someone to feeling creeped out)

Exactly my thoughts when I first saw the comments!

Reply on news.ycombinator.com