Hacker News Reader

Comment by thamer | Hacker News Reader

Is it very difficult? So many of these spam accounts follow the exact same format, with names following some variation of "Free stuff Telegram: +1234…" or "s3xY p1cs c0mE 2 mY chAnNeL", etc. Tons of them post these obviously made-up threads about how "Dr John Smith" has helped them triple their crypto investments.

There's probably a dozen or so templates that many of these spammers and scammers use, and if you can spot them in a second, it's not hard to train a model to recognize them also.

It is out of control because YouTube does not care to fix it, not because it's some insurmountably hard problem that one of the top AI companies in the world just can't figure out.

View on HN

I can totally see that if you assume it’s simple and easy to fix that it must just be another case of people not caring. Yet here we are replying to a post about the not-ideal steps the company is taking. And further this is a problem for multiple large companies with user commenting and chat, from Google through Roblox, Twitch, Twitter and beyond. There are a few things that make naive solutions impractical:

* Cost to build them.

* Cost to keep them up to date in a highly adversarial environment. Note this might mean scraping the solution and starting again.

* Cost of running them at huge scale.

Particularly important is to understand the attack surface is so high and cost to the opposition is so low that people will defeat your approach recreationally just to spam obscenities.

Guy makes concrete suggestions, you respond with abstract rebuttals that add up to "it's too hard". All he's saying is low-hanging fruit is pluckable; pluck it.

Guy doesn’t understand why perceived low hanging fruit hasn’t been plucked and I explained why it’s not that simple or low hanging. That’s fine, naive filtering (now with added ML!) is the first idea everyone comes up with. Then they learn about the Scunthorpe problem and beyond.

> Guy doesn’t understand why perceived low hanging fruit hasn’t been plucked and I explained why it’s not that simple or low hanging.

I'll explain too; Money. YouTube make money from the spam accounts, As twitter does, Reddit too. Spam is a very profitable market.

How does YouTube make money from spam comments?

Pay Google, Google turns a blind eye.

Why else has Google not cracked down on spam accounts? It has the tech, it knows it has a problem and the best remedy they can come up with is shortened YouTube usernames? There's more to it behind the scenes.

A real example is that I worked for a business where my job was to sell email addresses gathered from soft-core pornography websites. A batch of working emails could easily fetch £500.

Reddit will sell you accounts for your "campaign" if it profits reddit and it's all the same for any big walled garden.

Spammers contribute to user & engagement numbers and might even drive real engagement behind themselves as (angry) real users respond to them.

Gmail, Twitter, Facebook and Reddit have all but eliminated spam comments, yet they are everywhere on YouTube. Regarding scale: this is the company that scan every second of every video uploaded for copyrighted material, I'm sure their servers could handle a couple of (channel-configurable?) regular expressions per comment. Regarding Scunthorpe: they clearly don't care about false positive flagging when it comes to ContentID and video reporting, so it's definitely not what's stopping them here. And if they suddenly do, they have the resources for human review and appeals like everyone else is doing. Regarding low-hanging fruit: a handful of individuals have done this themselves already and it works great [0]. I think Linus Media Group started using this this on all their channels and the difference has been night and day.

[0] https://github.com/ThioJoe/YT-Spammer-Purge

Sure, it's hard to get around sophisticated operator., but most of the spam on YT that I see is not sophisticated.

You wouldn't even need ML, for the majority of them.

There is actually OS tool[1] that can do this for individual creators. That tool, finds a lot of spam, that YT itself does not, and it's developed by a single guy.

So there definitively is plenty of opportunity to pluck the low hanging fruit.

But Goolge has probably 100 of PHD's developing some kind of ML, that ends up performing worse than some simple regexes, because you don't get promoted for simple solutions.

[1]: https://github.com/ThioJoe/YT-Spammer-Purge

> Then they learn about the Scunthorpe problem and beyond.

I comment pretty regularly on youtube to get random thoughts out of my head about the video I'm watching (I'm under no delusion that someone relevant will see my comment), and all the spam replies I get use unicode numbers in the username to share phone/telegram numbers. Circled numbers [0] are a favorite, though not the only ones they use. The Scunthorpe problem doesn't seem very relevant here.

[0] http://xahlee.info/comp/unicode_circled_numbers.html

How do you know the low-hanging fruit hasn't already been picked, and that you're not just seeing the tip of the iceberg when you see spam?

There is a community project out there to filter and delete most of the spam and many creators use it. It was developed primarily by one person in their spare time. Google has no excuse.

To be fair, Google has tens of thousands of developers, and so would have a hard time being anywhere near as productive as one person.

^ this might sound like a joke to many, but is hauntingly accurate.

The common view on HN seems to be that google products optimize for promotions, and not even executive or entry level promotions, primarily middle management promotions.

I don’t think it’s entirely true as hopefully the finance department has folks smart enough to realize if headcount is adding negative value or positive value.

I wonder if the problem is sheer technological difficulty or if it's similar to antibiotic controls to prevent resistance development.

It is an adversarial space, the reason the names look ridiculous is because they have evolved to escape pattern matching as it also evolved.

YouTube and Google are notoriously unhelpful when it comes to instances when they catch a human in their automation drag nets. They lack the capacity to support their platforms so they are probably being cautious not to create support and ill will they can't handle.

It’s hard, if you’re insisting on doing it the Google way i.e. ML+AI with zero humans involved.

Having a person keep an eye on the top 50 popular channels and spotting the patterns in commentor names would be very effective, but having a human is not Google

Exactly this is my guess too. Literally everyone is seeing these very obvious spam comments everywhere. Have a few people monitor comment sections manually and then maybe one guy who goes over the spam ones and come up with a regex-like matching rule to detect these and move them to the "potential spam" section of the creator's dashboard. It's gonna be whack-a-mole but it's rather responsive. I get it, they want to automate everything, but evidently that only gets you so far, your model will never be at 100%, and we didn't even get to false positives yet.

That means you're now selecting for all the spammers that are somehow evading the eyes of that crew. It doesn't matter whether they are doing it intentionally initially, this un-natural selection will inevitably lead to spammers figuring out how to not be seen by those Google people.

It sounds like you're assuming that Google sticks to a single approach and doesn't waver from it, despite spam still being present. They wouldn't do it like that, though, would they?

The point is more that, unless Google keeps continuously expanding their approach along every dimension, whichever spammers happen to be evading them at any given point will stick around and multiply.

The thing about spam is that as it becomes more evasive, it's also less clear for the humans it was intended for. Once they raise the bar high enough, the spam will be gone.

But what are you suggesting then? Not even trying and accepting the fate of YouTube drowning in spam? Keep chasing the dream of the perfect spam fighting AI that's been just around the corner for the last 5 years?

Incidentally LTT just commented on the announced changes and rants about the prior situation at 15:20, for some insight from a Creator's perspective.

https://youtu.be/hf5N2UVGp-8

Yeah, it's not working though. I bet an engineer that trolls the comments for spam every day, writing one manual filter a day would be pretty effective.

Same for Instagram and Twitter. The formats of the obvious spam is super simple to detect on both platforms for years.

Yet I haven't seen ANY progress on either platform. Assuming that they are much more competent than a random dude like me as a whole tech company, the only possible explanation is that they have incentive not to prevent spam.

Perhaps: spam = more notifications delivered = more app opens = more potential feed engagement since they've opened the app even if from a spam notification = more ads shown = more revenue = happier shareholders short time... all at the expense of losing long-term trust to the platforms.

Have you been working on actually detecting them or are you assuming they’re easy to detect? Did you go digging for false negatives or false positives in other languages? Considering that not all special characters are printable, these platforms don’t limit their users to ASCII, random foreign words are common in other languages, the false positive rate has to be pretty much zero, etc etc etc I’ll bet it’s a whole lot more sophisticated than you imagine.

I'm assuming they're easy to detect yet I'm super confident about the patterns I'm seeing that I'd bet money that it can be implemented.

False positives should ideally be zero yet algorithms don't have to ban right away.

If there's a relatively small team of people reviewing those spams manuallu after the algorithms flag, I don't see any problems with the approach.

I of course get there are many different corner cases, Unicode characters that look like other letters etc. to bypass the filters yet they're also very easy to implement a filter that would probably detect many spams right away even with many of the corner cases. But again, a human team would still review these tweets anyway giving no room for false positives.

About 90% of the public Youtube spam I see on YT still follows this ultra-simple template: one person asks something on the "top comment", another one replies with a faux-legitimate message.

The question is often "does anyone know how to pirate videos?", "...how to invest in crypto", "...how to get fake instagram followers", life coaches, etc.

It's not really custom messages with special chars, it's something that I could detect by using Ctrl+F in the browser. It's not something they'd need special tools on their end to catch. The only thing that is changing is the name of the users.

So yes, I'm gonna agree with GP that "the formats of the obvious spam" are super easy to detect.

> if you can spot them in a second, it's not hard to train a model to recognize them also.

something Minsky vision something (I know it's an urban legend but still, it figures.)

>There's probably a dozen or so templates that many of these spammers and scammers use, and if you can spot them in a second, it's not hard to train a model to recognize them also.

I'm betting it's probably not hard the spammers and scammers to switch templates if old ones get caught.

I think a company like Google have the resources to dedicate a team to work on that permanently, but as I suspect, working in something like this doesn't get you enough merits, so it's better to work on a new shinny useless feature than an actual useful but not shinny feature.

They don't even need to change the name. I recall seeing a giveaway livestream scam in which the hacker didn't change the name of the channel at all and it was still bringing in btc.

thаmеr is not thamer.

The space of all unicode characters is too big for you to look for every single possible "looks like" pair.

Unicode, as unpopular an opinion as this is, should only be used for presentation and not storage. Anything past ascii is a security vulnerability.

> Unicode, as unpopular an opinion as this is, should only be used for presentation and not storage. Anything past ascii is a security vulnerability.

While at it, we should also ban the letters I and L as they can be confused. People who have those letters in their name can just request a name change. /s

Leave Unicode alone for the vast majority of the world whose language isn't actually writable using ASCII.

https://en.wikipedia.org/wiki/File:QWERTY_1878.png

Notice some keys missing there?

The difference is that we have a few good quality fonts, like liberation mono, which let you tell the difference between l and 1 and 0 and O. There can't be a font which does that for the whole of unicode space because a and а are by definition the same glyph.

What's your proposal for differentiating between mais (but) and maïs (corn), or between café (French, Spanish) and cafè (Catalan) using ASCII?

How would one write an ideographic language using ASCII?

> What's your proposal for differentiating between mais (but) and maïs (corn), or between café (French, Spanish) and cafè (Catalan) using ASCII?

Context, but...

> How would one write an ideographic language using ASCII?

... We're talking about security sensitive context. Users can write stories with all the Unicode they want, just don't use it for identifiers like user and channel names. Reddit got it right.

So channel names like "le canard enchaîné" or 美味しい食べる are completely verboten from your perspective.

Do you know who has it even worse? Spotify

Because of course this band from "part of the world" has the same name as "other band from a different part of the world" or even from the same country and they have nothing to do with each other (because why google before naming your band - but I digress)

You don't need a human to even look at it until after an "AI" has already flagged it as "something fishy" worthy of human attention, and even then the font don't matter so much, because you'd be using some fancy scripting or tools to be pointing out to the user what exactly is potentially "fishy" about it. Seemingly insurmountable task becomes common "drudge-work" with the aid of some cleverly scripted tools. It's actually what computers are good at. Automating and simplifying repetitive tasks over large data sets. Something that Google's people should be well aware of and able to fairly easily implement. You could even script the tools such that the humans tasked with checking on the flagged materials would be helping further the training of the AI through their responses to the individual situations for less false positives, and more accurate flagging going forward, making the humans' jobs easier as they continue to do it.

Now you've just made name changes unbearbly slow for everyone to supposedly make like better for people who already have access to a keyboard with latin characters on it. If people can manage maths with greek letters they can manage logins with latin letters.

You can like, just normalize the input on the backend when scanning for spam.

Every spamfilter I've ever used that's worth a damn does that. Sometimes you get slightly odd syntax (I once configured one that replaces A with a 4 so it could detect common fake uses), but this is a solvable issue.

No, foreign names do exist. they problem is how to disallow confusables then. and this is solved by UTS 39, without needing to compare against all existing names. https://unicode.org/reports/tr39/

and use the secure subset of UTS 31.

you can eg use my libu8ident, and please dont use the broken confusables list.

Render to pixels. If they look about the same, but are different, that is a high signal for spam.

Ноw wоuld уоu knоw whаt уоu ѕhоuld bе rеndеrіng аgаіnѕt tо сhесk аhеаd оf tіmе?

The majority of the above sentence is not valid Latin characters for example.

All of the usernames, to detect lookalikes.

That's an n^2 problem. For every new used you'd need to compare them to every already existing user. Do you think you ought to be charged $5 for every attempted registration?

>Mole is too close to Moie, please select another user name. Your bank account has already been charged.

It's not n^2. It's a lot better than that.

    for i in users:
        for j in users:
            if same_as(i,j):
                reject(i)
            else:
                accept(i)

That is literally the definition of n^2.

Reply on news.ycombinator.com