Hacker News Reader

Comment by nickff | Hacker News Reader

Your explanation seems purely speculative; do you have any reason to believe that it's actually true?

I mean, different providers have different features that attract different types of people. But if you want quantified data, there's at least this that I found with a quick Google, showing a significant difference in age demographics between providers:

https://www.statista.com/statistics/547531/e-mail-provider-r...

And there's quantified evidence that older people tend to be more conservative:

https://www.chicagobooth.edu/review/there-are-two-americas-a...

So that alone is worth taking into account.

It might be true, but how likely is it that users are more important than the spam detection algorithms? I don't think this user data is sufficient to exclude/disqualify other explanations.

"...how likely is it that users are more important than the spam detection algorithms?"

I'm not sure you understand how spam filters work. No one is sitting there hard-coding all the possible terms and phrases that indicate spam; that would be unfeasible. The spam filters learn what content is considered spam based on user activity and user flagging. It's not that users are "more important than the algorithms," it's that the algorithms are trained by the users in the first place.

In more general terms, the algorithms represent user preferences, so the user demographics directly determine the algorithms' behavior.

>"In more general terms, the algorithms represent user preferences, so the user demographics directly determine the algorithms' behavior. "

It all really depends on the algorithm used. I am familiar with many algorithms which could be used in these cases, and I'm not sure whether Google is using a 'stock' algorithm, or as I suspect, something tailored to their requirements. Each algorithm will produce different results, so which one you choose has a huge impact on what is 'filtered'.

The user-base has a substantial impact, but you could probably get a variety of results by applying different algorithms to the same user data. You implicitly asserted that user demographics were the dominant factor, and I am not sure that's true.

Different algorithms do indeed learn from data differently. But the entire purpose of a spam filter is to filter out spam, a word which is defined subjectively. Any spam filter which acts contrary to its users' collective preferences would be considered bugged and broken, regardless of how it decided that, because it would mean it's not doing what it was designed to do, and not learning correctly.

Gmail spam quality is internally evaluated based on the ratio of moves into and out of the spam label. So you're completely correct: it is optimized for how its users collectively define spam. There is no objective definition of spam.

The spam detection algorithms are trained ON the user's reports, so it's certain.

Which algorithm you choose, what data is used for training, and how the data are used are all very important. The user-base has an impact, but I am not sure whether it is the primary cause of the discrepancy between e-mail providers.

Another very scientific study on the subject: https://theoatmeal.com/comics/email_address

> https://www.statista.com/statistics/547531/e-mail-provider-r...

Why does this study exclude ages 35 to 55? Seems like a big gap to me. So this study is only comparing Zoomers, Millenials, and Boomers, and completely excludes Gen X.

I was curious about this, I have 2 conservative emails in my spam folder, one sent from a colo facility, and one sent from amazon ses. Both seem to have adequate dmarc and spf records, so I can only imagine that it'd be something in Google's filters.

Personal anecdote: I've given my details to both Republican and Democrat parties here in Texas. The only email list I ended up on the Democrat side was a local congresswoman's mailing list (but I've been phone-banked a ton). On the Republican side, I've wound up on 2020maga email lists, something called "middleamericanews", Texas GOP, something called "americasfreedomfighters", etc. A bunch are really low quality e-stores. I can only surmise that (at least here in Texas), there are differences in how the two parties partner with their mailing lists.

I imagine IP geolocation probably plays a role in where these groups send emails in the first place. I'd guess someone with an email account most accessed from Texas would be sent far more conservative emails than, say, someone with an account accessed from California or Manhattan. But this is definitely just my own speculation.

https://www.statista.com/chart/9937/the-gmail-yahoo-mail-age...

https://www.pewresearch.org/fact-tank/2017/03/20/a-wider-par...

Experience, perhaps?

That maps partially to my own experience. The majority of folks I know who identify as 'conservative' are older, less tech savvy, and tend to use whatever they had set up a long time ago. @AOL? @Yahoo? @MSN? If it's working, they'll just keep using it.

A few I know have made a transition to gmail, I think at the behest of younger children (as in, they may have set it up for them during a holiday visit, etc). But in my own life, with the majority of conservatives I know where I also know their primary email, it's not gmail.

My experience agrees with you and I just realized that you can roughly guess someone's age by their email domain. As a Millenial, I've used gmail for going on 20 years, but I feel like that was more or less the peak of email - I don't know that Gen Z uses anything else. My guess would be they moved onto social media and while they probably have emails, it was never their primary method of online communication.

I would wager that for most Gen Z-ers (and younger), their emails are used only for two things: confirming account registrations and communicating with older people (a la teachers, businesses, etc.) Most of their inter-demographic non-realtime communication is likely social media and texts.

I would imagine the demographics are different just purely based on the fact that Yahoo Mail has been around for longer than Gmail.

Reply on news.ycombinator.com