Social Media Has Failed Its Self-Driving Test
bloomberg.com
bloomberg.com
There are thousands upon thousands of rendered, cartoon, and live-action videos featuring popular childrens characters like Spider-Man and Elsa that have extremely disturbing themes dealing with punishment, handcuffing/restraint, dismemberment, injections, bodily waste, etc. It seems like they get more and more unusual as autoplay goes on. As a parent they are extremely unnerving and I'll certainly never let young children watch youtube unattended and I warn every parent I know about them.
There are places for curated content and there are content bazaars. YouTube Autoplay is a bazaar. Facebook is a bazaar. Recognize where you are, the motivations of those putting things in front of you and your family, and act accordingly.
Today on HN there was an interesting discussion on addictive apps - https://news.ycombinator.com/item?id=15662004
In essence you have parents vs the best funded organizations in the world, packed with PhDs working full time to grab all our attention, children being the most susceptible. The only advantage parents is physical proximity. Disadvantages include lack of time, knowledge, awareness etc. I'm not betting on parents winning
Yes, it's up to the parents. But unless you have time to pre-screen every minute of content your kids watch (hint: if you have a job, you don't have time to do this), at some point you have to trust a ratings agency like MPAA, ESRB, or a company like Google when they say - "this content here, it's safe for kids". The obvious problem is that YouTube Kids is utterly failing at this.
What's the realistic alternative? Watching hours upon hours of kids content on YouTube and creating your own playlists? How do you stop a child from going to a "recommended video" afterwards? What about ad content, how do you filter that?
Edit: The only realistic alternative is not letting kids watch YouTube. This is fine, but then maybe Google should not be advertising a section of YouTube as "for kids".
And even then, things get pretty sexual.
I would argue that if your kids are watching so much of this stuff that you don't have time to monitor it, they're watching way too much of it. "Hours upon hours" is definitely way too much.
Your kids will survive even if they're not plugged in to streaming media 24/7/365. Really.
It never ceases to amaze that the kind of psychological engineering these companies engage in is outside the bounds of any established ethics, but somehow they escape unscathed because money.
I've been confused reading all these stories about YT Kids, how do the kids even get to this stuff? The obvious search terms are banned by the app, but how deep down the rabbit hole does one have to go?
This garbage on YouTube Kids is aimed at getting children to watch it and intended to be disturbing to children. The titles say things like "educational, learn numbers" and the channels are called stuff like "Kids TV."
Yes, and any sane parent would agree. Physical safety is a prime consideration.
Here's a bunch of examples (query: fork in electrical outlet):
https://www.youtube.com/results?search_query=fork+in+electri...
(Come to think of it, college is when you really have to worry about your kids frying themselves because they saw something cool on the Internet. At that point, you've let your guard down and assume you don't need to babyproof. But I distinctly remember being sad that I'd missed the exploding wires demonstration in physics class and deciding to replicate it myself with a frayed Ethernet cable in my dorm room...that could've gone very badly. And my friend electrocuted himself trying to get pirated satellite with a board he'd bought off the Internet and a dish we self-installed on the dorm roof.)
PTSD from seeing horror movies at the age of 2.
Sexually precocious behavior at 6 due to having seen hardcore pornography at the age of 3.
It's hard to tell what will or won't deeply affect a child and even harder to gauge the specific ages where things might become appropriate. It's really per-child, and no matter how well a parent does gauging that, mistakes will be made.
There's no way this can be automated, as it's never going to be one-size-fits-all. The combination of generally sanitized media channel + parental supervision will (and should) endure, IMO.
Yes. See the number of perforated ear drums after the "You've been tango'd" adverts.
https://en.wikipedia.org/wiki/Orange_Man_(advertisement)#Bro...
(the lead says the ad was banned; his section says it was voluntarily pulled. Either way it could have been banned: any system of media rating (eg advertising standards or BBFC) include imitatable violence.
And I don't think a parent's discomfort is sufficient, either. There were things that I watched as a child, with my parents' knowledge, that I still consider to have caused me trauma.
For some things, the potential hazards (in this case, lifelong desensitization to, or increased propensity to violence) so drastically exceed the potential benefits (in this case, a bit more profit and less work for Google), that we should proceed with the precautionary principle, rather than just saying "F-it, show em whatever, we don't yet have proof that it's bad".
Learn a bit of perspective.
I know it's the "think of the children" thing, but I hope Youtube sees the outrage about these spammy kids channels and does something so they aren't suggested and autoplayed. To me that's a special case. Grown adults who see fake news on Facebook and take it as the truth regularly.. Is that really on Facebook to fix that?
In the past few years, I've been learning intensely about communities and family: got married and learning to be a step-father. I am also involved in a legal-tech startup and got to hear how lawyers think about governance and law. I don't think social media is a substitute for community either, although it can be beneficial _if_ used within the context of community.
Here are some of the things I learned:
- There is a tension between global and local concerns. Neither alone can address everything. Local concerns -- communities -- have people living there. They have a stake in their local environment. There are issues when people outside of a community (people without stakes within the community, who are not residents) try to influence things. This could be anything from coordinating regional concerns (such as people building upscale neighborhoods in a flood control area built by the Army Corp of Engineers in Houston), or outside investors holding municipal bonds.
- I discovered Christopher Alexander's work on pattern languages and designs. I found his keynote to the OOPSLA '96 conference. His work inspired the design patterns of OOP as well as Human Interface Design, but I don't think his vision and philosophy was ever fully implemented. His life's work somehow addresses how to design for both global and local concerns. The idea is that architects develop pattern languages that will always result in cohesive, livable designs (global), and the actual users of that pattern language are the people who live within that community (local). Although this is primarily focused on building design, I think it has some overlap with governance. Physical environment will shape and influence how people interact with each other.
- There are people who are isolated because they don't fit with their local community's norm. Allowing them to connect with other people around the world is great. However, I think it is important for a person's psychological health to have some connection with their local neighbors.
- My wife and I tried allowing my step-daughter to have a smart phone, mainly so we can reach her in an emergency. She quickly found ways to become absorbed into it, neglecting her responsibilities to the family and our expectations for her as parents. We tried using more tech to restrict it. She got around them -- claims innocence (and to this day, I don't really know whether she was or not). In the end, we got her a flip phone. She called a "granny phone", and we blocked calls and texts at the carrier level. She doesn't have her own tablet or computer anymore: we have family devices that are kept in the family office and she is forbidden to take it in her room. (Where she still sometimes sneaks it to bed and stays up all night watching youtube videos).
- My wife told my daughter that she had to be at least 13 to get on social media, and she is forbidden from it. That was half a year ago. I had always wanted to get my daughter to _earn_ her smartphone (and by extension, social media) through some sort of modern-day rite of passage. But over the months, I am more and more inclined to forbid her from social media until she graduates high school.
>YouTube Kids has tons of fun and educational videos that are just right for kids. There’s also a whole bunch of parental controls that let you create an experience that’s just right for your family.
>CONTENT ON YOUTUBE KIDS
>Our app is designed to filter out inappropriate videos for kids, but no system is perfect. If a video that’s inappropriate shows up, you have the power to block it, flag it, and bring it to our attention for fast review.
So, it attempts to be "right for kids" and "filter out inappropriate videos". Is that working? I'm not quite sure.
[1]https://play.google.com/store/apps/details?id=com.google.and...
[1]: https://medium.com/@jamesbridle/something-is-wrong-on-the-in...
That said there is a MASSIVE difference in a poorly animated earworm like the "Finger Family Song" and Chase from Paw Patrol being decapitated in a strip club, which is a rarity.
Plausible scenario is that it fails is that they assign a few human reviewers, and set the threshold of reports required so that those humans aren't overworked. Then they make the erroneous assumption that their staffing for the task is adequate and that items falling below the reporting threshold must be fine. Wrong.'
I believe the discussion here about parenting is a diversion from the real issues at play.
This is not a ‘won’t somebody think of the children moral panic’. Instead, we’re feeling a foreshock of the problems we’re going to experience, caused by the unintended consequences of the systems we have built - and what that means for us as a society.
YouTube videos for kids is a lighthearted entry to the subject - we could be discussing news, porn, education, whatever, and the problems and implications would be broadly the same.
The key issues raised here are: * The ‘delamination’ of content and author and how that affects the awareness and trust of its source. If, for example, a scientific paper and a bunch of woo is presented in the same way who can tell which is more legitimate? ‘Just teach critical thinking’, or ‘it’s called ‘parenting’’ are not acceptable solutions as by the time enough people have been taught to provide herd immunity we will have long since succumbed to this pandemic of bullshit.
* ‘...the impossibility of determining the degree of automation which is at work here’. If both humans and machines are creating content tailored for every possible niche, interest, fetish and keyword combination, and algorithmic personalisation makes it possible to exist entirely within our own personal tag clouds what does it mean for us as a society, which requires a basic set of shared values to function?
Junk content and the pandering to base instincts is not new, but our ability now to automate the creation and dissemination of such content to pander to every possible interest and combination of interests at vast scale is new - and we do not yet have the cultural toolkit to deal with it.
I wonder whether in a few years time, once we’ve really felt the impact of all this, people will look on today’s enthusiasm for putting ‘social’ in everything with the contempt and horror we do with last century’s enthusiasm for putting radium in everything.
Something Neal Stephenson remarked in Diamond Age: this kind of tech is not a substitute for parenting.
Social media (even YouTube) is mostly intended to entertain. If it turns out to be bad at other stuff, well, that just means we have to create systems that are. Super popular web sites are not going to be any more socially constructive than the sort of magazines you find at the checkout aisle of a grocery store. Nothing has really changed here.
If the market/technology does exist where someone else could do a better job then there would be many alternatives, it's not like Youtube Kids has some lock in mechanism.
PS: the self-driving moral decision comparison in the article seems baseless, I very much doubt anyone developing self-driving algorithms are considering "moral decisions", there is no moral decision to be made, the car will try to stop (in its lane) as fast as it can if it detects an obstacle, it won't try to swerve (after performing some imagined moral decision algorithm) because that would be illegal and it potentially increases liability of the software manufacturer.
This Bloomberg piece on the other hand simply links to Bridle’s blog and says “Yeah! What that guy said!”. Really thin “content” imo.
Not referring to the person of the author, but people use tablets and phones as stand-by buttons for their kids. However much active the kid might be, they get parallised watching cartoons and whatnot. But in a recent interview with a local doctor from a working-class neighbourhood where I live, he was referring how excess use of this is causing kids to suffer from under-developed motor skills (Evrensel.net on the Esenyalı neighbourhood).
Social media is media, but it's also social. A lot of people are trying to apply the standards of traditional media to something that's fundamentally different. I'm not sure where on the spectrum social media lies between "private conversation" and "network TV", but I don't think that we can exactly apply the social norms of either.
The vast majority of people would agree that some degree of monitoring and censorship is necessary for the prevention of crime. Beyond that, I think we simply have to accept that some amount of content on social media might offend us.
Social media is created by people with a very diverse range of beliefs, opinions and standards of decency, so it's unlikely that we'll establish a universally agreeable set of standards. Social media companies have a limited ability to control the content on their sites reactively or via algorithms, but it's logistically impossible for them to consistently enforce their terms of use across all content.
Many voices in the traditional media would have us believe that occasional offence is a fundamental flaw in social media, but I think that's a bogus narrative.
https://www.statista.com/statistics/259477/hours-of-video-up... https://research.fb.com/facebook-s-top-open-data-problems/
Maybe people willfully choose fake news on Facebook because they're tired of the fake news the mass media broadcast.
"Oh but those are wrong choices and somebody needs to put a stop to it!"
This is just a push for autoritarianism. No thanks.
We live in a market-driven society, people are free to choose to consume what they want to consume (as you point out) but that doesn't give carte blanche to the producers in the marketplace to do whatever they want in the name of ad revenue. When it does reach a tipping point, as I think we're seeing with this Medium article, regulation will start to come to a niche industry that had previously operated under the radar. Especially when human health consequences are involved, the government steps in to ensure protection to its citizens (and its most vulnerable members, children).
If you really are upset that you won't be able to view algorithmically generated children's cartoon videos stuffed with tropes and keywords that exist for no other purpose than to make money, I'm concerned for your wellbeing.
We have gotten to the point where even reasonably conscientious and responsible adults have trouble raising their children. When the advertisers make money by making it a harder world to raise children in, they are externalizing their costs and internalizing the profits.
Some amount of externalities we allow due to pragmatism (the cost of enforcing the regulation is greater than the benefits), but when the line is crossed, some sort of regulation is required.
It is a reasonable policy question to ask where the line is drawn, and what sort of regulation (on the extereme authoritarian side, it is drawn very close and uses criminal law, on the extreme libertarian side, it's drawn less close and might use tort law), but outside of pure anarchy, it's not reasonable to say the line need not be drawn.
Edit: P.S. Where was the personal responsibility of Facebook after they were caught peddling propaganda placed by foreign actors? As I recall it took a lot of arm twisting to get them to admit there was even an issue.
Government on the other hand makes the decision and forces it on everyone. Not just those who elected it, everyone.
Edit: as for the foreign propaganda, how many nations and what interests were/are they peddling? If it's already illegal then sure, it should be forbidden from the network. If it's not, then I'm fine with it.
Please, tell me how I can escape Google? God knows I've tried - I have a Fastmail email account, I use DDG, and I stick to Apple hardware. That doesn't change the fact >90% of advertising I see on the open web is still "suggested" for me by Google's algorithms, that my employer uses Google Apps and Google Drive for just about everything, that many links I encounter on social networks are AMP, and that most of time I need to verify I'm not a bot I'm filling in a recaptcha.
Saying I have a choice not to use Google is only true in the academic sense. It's theoretically possible, but not without extreme effort and probably not without finding a career outside of the tech industry.
What you don't have is the power to make others choose what you'd want them to.
Actually ironic is not the most accurate word. It's telling. This is another article which is essentially a call to censorship. And what's to be censored? Well like most of these articles, that's not really discussed beyond whatever the author happens to disagree with. You see, jamming in misleading tags to produce hits isn't a problem when it benefits the author or their publication, but when others do it? Oh, it's time for a serious conversation now.
That said, there is a role for government regulation. Specifically, to break up monopolies such as Google.
Then, when there is proper competition, you have some freedom to choose the platform that’s policed by algorithms that you agree with.
> A step back to assert human control -- even if it cuts into the tech companies wide profit margins -- is overdue indeed.
The author seems to think these problems cannot be solved with technology. Maybe they think that, or else Facebook, Twitter, et. al., would have solved it already?
I think that it is far more likely that our society directly incentivizes the companies to not solve the problems. That is why they are not solved. Because we lavish them with money to leave them unsolved.
Legislate that political news must be regulated. Legislate that companies that exert social control over a population, like Facebook, must do so responsibly. Don't just tell them that humans must curate all content. That won't work. Humans are too expensive and corruptible anyway. The solution is structural in society just as the problem is. Take away the profit incentive for big tech firms to cheat and feed lies, and our society will improve directly in line with what the author probably wants.
This is not as simple as it sounds, of course, because the regulators would also have plenty of incentives for bad behavior.
I actually think that government regulation can be tremendously useful in certain circumstances. But it's not a magic solution, and you have to be clever about it. In the particular case of political news, the potential downsides of giving too much power to regulators are at least as dangerous as giving too much power to Facebook or Google.
In what way? Establishing fact-checking institutions or empowering the FCC to fine broadcasting of false political speech? Or with libel laws to allow people to sue for false allegations easier?
> Legislate that companies that exert social control over a population, like Facebook, must do so responsibly
In what way? Who what agency determines what is a responsible way to "exert social control over a population"?
Do we need to be reminded that these are opt-in services where you agree to a ream of terms before being allowed to use the service?
At which point does an online service go from being simply a place for people to interact to a company that needs regulating? Does HN qualify or do you need XXX million active users?
Legislation should be a last resort since it is so easily circumvented (see tax code) and unfairly applied (see criminal code, specifically the section on drugs).
So what you're saying in our current political climate is "Change the rule of web based business every 4 to 8 years dependant on who gets elected"?
In the US we already know this doesn't work out because "big tech firms" will just buy the lobbying they need to get what they want. At best we can put some bandaids on the internet, at worst we're making it some kind of authoritarian nightmare depending on whatever companies get to enforce contracts based on the law.
Not just no, but HELL NO!
You may be imagining that the regulators will censor the news in a manner that conforms to your opinions.
That's not a safe bet.
It's extra work, of course, to put substance behind what you're saying, and that isn't always an option. But refraining from posting is.
Versus a real problem -- oh shit, we accidentally created a brand new Internet where there are no more editors, everyone is free to find their own content democratically, hooray, but some of the people we put in charge of creating that content are hyperactive amoral entities.
I know a bunch of Facebook employees have this view of the whole "filter bubble", "fake news" situation which is, more or less -- Deal With It. The news feed algorithm is here to stay, it's the new printing press, editors are obsolete, and everyone gets to decide their own truth now. That view is evinced ambivalently at e.g. https://www.wired.com/story/the-solution-to-facebook-overloa... .
But I'm not sure how that view copes with the YouTube monetization model which .... starts with children's videos ... goes to ad revenue ... and leads to children being videotaped apparently in real distress for money.
Here are 2 points of reference:
(1) YouTube channel "DaddyOfFive" had 750,000+ subscribers and featured what was fairly clearly child abuse ("pranks"). Somebody finally thought it might be a good idea to report this to authorities and the father lost custody of some of the children. https://www.washingtonpost.com/news/the-intersect/wp/2017/04...
(2) The comment at https://news.ycombinator.com/item?id=14910125 links to an article at https://vigilantcitizen.com/moviesandtv/something-is-terribl... which provides some evidence that toddlers' content preferences are influencing the creation of new content which involves, as part of making the content, putting children in distress. This seems to cause actual measurable harm to children.
The news.ycombinator comment said in part :
> The feedback loop is so tight that many of the creators have converged on designing some of the most visceral (and disturbing) material that appeals to kids at a reptilian level. Think about how genuinely weird little kids are. Sometimes they do stuff that would be scary if it wasn't just a minute of innocent pretend and a toy. Stuff like kidnapping each other or performing surgery on each other. Well, now there are hundreds of thousands of YT videos illustrating that kind of material in fine detail. Many of them have millions of views.
And the sample videos' captions include:
> Many videos feature the girls screaming or crying their heart out. In this video, this girl has something in her mouth that tastes horrible … And she’s forced to keep it in her mouth … And she doesn’t seem to be acting.
> Here, a girl takes something from the toilet and force feeds it to the other girl. She doesn’t like this.
> In another video, a creepy dude with clown makeup barges into the girls’ house and starts grabbing them while the girls scream and attempt to resist him.
Is this wrong? Should this not be happening? Is this anything but a natural effect of the 2010-2011 "filter bubble" and the 2016 "fake news" phenomena?
Further, the fact that YouTube, Facebook, et al. Have failed to police content to the author's satisfaction is not indicative of a failure in AI - especially when these corporations are effectively incentivized by view counts and advertisement not to remove these videos.
In short, YouTube's failure is not a valid indicator of the effecticeness of AI in general.
Pretty sure he does, from the OP:
> A step back to assert human control -- even if it cuts into the tech companies wide profit margins -- is overdue indeed.
The tech companies use these algorithms because they're cheap and they're good-enough for their profit-making purposes. If Google's algorithms can't keep YouTube Kids kid-friendly, then they need to hire the human moderators to make up the difference or kill the product (because it's false advertising).
As to whether today's AI is up to the task, I think it's fairly hard to say definitively. It seems certain that we haven't yet fielded a system that does it very well though.