Bypassing Gmail's spam filters with ChatGPT | Hacker News Reader

Bypassing Gmail's spam filters with ChatGPT

neelc.org

207 pointsneelc120 comments

Using these tools to mitigate spam is as likely a scenario as this, imagine that every spam mail receives a masterfully crafted response showing utter fascination and interest in SEO, or helping out a Nigerian prince. Every phone call to an unregistered number is answered by an artificial, frail, and forgetful lady that is trying her best to register gift cards.

When reporting an e-mail as spam it will not only block the address but waste the spammers time, rendering the actions unprofitable.

So the two AIs will be talking to each other, trying to suss out if the other is fake (a sort of Turing test), trying to con the other to keep talking or to really buy in?

Sounds absurd all right. But what's to prevent this from being the future of the internet?

Reminds me of Adventure Time Season 4, episode 10: Goliad

https://adventuretime.fandom.com/wiki/Goliad_(episode)

(Spoilers follow)

> Princess Bubblegum is concerned over her mortality, and reveals her designated successor to Finn and Jake. However, things get out of control when the heir turns against Princess Bubblegum.

> […] The heir is Goliad, a pink Sphinx with a mound on her forehead […]

> […] Goliad reveals that her mound concealed a third eye, and proceeds to psychically control Finn and the obstacles in the course for a perfect completion. She explains that with her in control, everything would be perfect. […]

> […]

> […] Finn and Jake confront Goliad but are beaten by her psychic powers. Goliad tries to read Finn's mind, and Finn narrowly avoids revealing the plan by interrupting his memories with nonsense. The new creature; another sphinx with an eagle's head, white feathers, and golden hair; rescues Finn and battles Goliad. Goliad tries to convince her brother, named Stormo, that they should work together, but Stormo refuses by screeching. They engage in a psychic showdown, but with their powers matched the two creatures are eternally locked in a mental stalemate.

Never thought I'd see Adventure Time reference on HN .

How about 2 AI developing a language to be able to speak to each other

https://www.imdb.com/title/tt0064177/

Oh my god a future where you are never ever quite sure if your online circle of friends are human or not.

I wonder if that inspires new platforms or types of tech that verify someone was typing it in vs pasting. But then do bots get better at typing it into the input boxes? Ugh.

And you're confident right now?

I… but… oh no.

My online friends circle includes three dragons, two coyotes, a raven, a squirrel, and a wasp.

All totally real. But then, I am a furry, I've met many of them IRL also.

I’d be okay with this option too. As long as people are who they claim to be and it doesn’t turn out that I’m in some horrible non-VR Matrix.

yes this is dog

At the same time, it's not necessarily pleasant to consider the prospect of an internet where 99.9% of traffic is generated by AI-powered spambots engaged in adversarial games with AI-powered anti-spambots.

But it does make for an interesting idea as a plot device in speculative fiction!

Cyberpunk 2077 features this reality. Its not discussed much, but basically only local small scale networking is done (like citywide scale) because the broader internet protocol has either crumbled or is polluted by adversarial bots. So its pretty taboo to go there or simply unused.

I'm sure someone else knows the lore better. There is a lot of supporting literature, just not a key component of any stories.

I think the lore is that the wider internet was intentionally taken down by a virus made by a master hacker in the past. Any attempts to connect with it gets you killed by either the virus or "feral ais", but some of the mega corps pay people to explore it every once in a while looking for stuff.

Honestly the most likely source of computers becoming self-aware.

That's basically the plot of the novel Avogadro Corp by William Hertling.

And a plausible way for them — on either team — to take over completely. Both treating us not even as pets but as grass. In this analogy the good AI are gardeners, the bad AI are cow farmers.

dead internet theory: https://dailyfreepress.com/2021/10/21/the-dead-internet-theo...

I thought we are already nearly there? I remember reading 90%+ of emails are spam and this was a while back

90% is a lower figure for how much of HTTP traffic is bots scraping sites, too.

Why is that bad? As long as it is invisible to humans and doesn't cause bottlenecks, who cares?

For one, it's a waste of energy.

Banks also waste energy deterring people from robbing them. Internet or not, we'll have to stomach a good amount of bad stuff in order to get to the good stuff.

If you haven't read Accelerando, I'll suggest it - https://www.antipope.org/charlie/blog-static/fiction/acceler...

https://maggieappleton.com/ai-dark-forest

The open internet will be a dark forest, beautiful but mostly populated by metal-brains, meatb-brains will retreat to their human only spaces, local Facebook or Instagram where you know everyone in person or WhatsApp chat groups.

Interesting - going to have to read this more in depth.

I also note that I missed it when it went past the front page at the start of the month. For any others who also missed it - https://news.ycombinator.com/item?id=34243709

Also the Reticulum of Anathem which was mostly full of spam-generators that were run by the spam-filter vendors to make their products necessary.

Btw, the chants from the book are real (and part of the interludes if you get the audiobook - which I recommend since the initially boring but ultimately important parts it aren't skippable without effort).

https://soundcloud.com/ztutz/sets/iolet-music-from-the-world...

The Thousander Chant is fun if you've got the right audio setup.

https://longnow.org/store/iolet/

> In Neal Stephenson’s new novel Anathem, the Decanarian Erasmus’ daily chore is to ring the Clock bells in his “Math” in a special sequence each day as he chants out the sequence. Those sequences and chants are all based on mathematical formulae that composer and coder David Stutz has put together into an album called Iolet.

I like the thousander chant but I only have regular laptop speakers. What do you recommend to really "get" the song? What would I hear beyond those super low basses?

A good pair of headphones would be my next choice.

It's not much "under" it - but rather the fill your senses with what it would be like to be in the Mynster as each nave did their chats in the great octagonal hall. And while its a passage about the Hundreders...

> We sauntered across the meadow to the Mynster. Even so, we got there in plenty of time, and ended up in the front row, closest to the screen. Voco continued ringing for some minutes after we arrived. Then the eight ringers filed down from their balcony and found places farther back. A choir of Hundreders came out into the chancel and began a monophonic chant.

Or the description of a Thousander chat

> ... As I walked toward it my perceptions cleared suddenly and I shook my head in amazement at my own silliness in having imagined it was an amphibian or a truck. It was plainly a human voice. Singing. Or rather droning, for he had been stuck on the same note the whole time I’d been awake.

> The note changed slightly. Okay, so it wasn’t a drone. It was a chant. A very, very slow one.

> I could have stood there watching and listening for hours. I got the idea—which might have been just my imagination—that {spoilers} was singing a cosmographical chant: a requiem for the stars that were being swallowed up in the dawn. Certainly it was music of cosmographical slowness. Some of the notes went on for longer than I could hold my breath. He must have some trick of breathing and singing at the same time.

Just let the sound fill everything.

Also in Reamde and later Fall. I forget exactly, but i think in Reamde they flood the internet with shit, and then some years later in Fall everyone has curated echo chambers of content, such that people signed up to different streams are basically living in different realities. Which i guess is just the logical conclusion of what we have now.

Cyberpunk 2077 features a world after this has happened. Not because it happened, its just after. If there is a reference to an older internet it features its disuse due to this.

I think the opportunity cost of better employee will make it hard for that to scale.

If you’re that good at scaling systems like that someone with legit traffic would be able to pay more than the spam companies.

Part of this is what I imagine is behind the “Blackwall” in the Cyberpunk universe.

There are still solutions in that case. Email addresses only allocated to verified humans, which only accept email from the same addresses or specifically allowed outside email addresses.

Alternatively, email addresses that are necessarily paid for that have maximum sending limits or extra costs above those limits, so those addresses are not profitable for spam bots to use.

I could imagine some kind of “super priority email” that costs a dollar or something, that gets prioritized and is very unlikely to be marked spam.

More ideas: Maybe the amount paid per email sent changes depending on the percent of previous emails marked “accept emails from this sender”. Maybe some percentage of the e-mail fee is given to the recipient of the email.

Note that if a highly spam resistant email account is adopted by everyone the amount of spam sent may fall significantly. Gmail doesn’t count because it still isn’t as good as a system where sending millions of spam emails is simply too expensive.

Every day we get closer to the Slug from Accelerando (by @cstross): a self-owned sentient corporation/419 scam

If as a (hypothetical) Nigerian prince spammer I get a "masterfully crafted response" from a mark, it would be obvious right away that I am talking to a bot. The kind of people who respond to such bait would hardly be able to write anything like that.

A "masterfully crafted" response to a nigerian prince spammer would probably sound a lot like a person who can barely write, possibly with what sounds like the beginning of dementia setting in.

Fun idea but impersonating someone else, especially your customer, sounds like a way to land in hot water. Also LLMs are not exactly cheap.

>imagine that every spam mail receives a masterfully crafted response showing utter fascination and interest in SEO

That would necessitate reliably detecting the emails as spam in the first place though. False positives in particular could be devastating. Imagine a chat bot coming up or going along with business proposals in your name for example.

You could do it with humans - every time you click "mark as spam" it doesn't just trash the email, it begins a long and drawn-out chatGPT conversation with the spammer, stringing them along.

I’m pretty sure most spam senders black hole any response, the money is in the target clicking a link and no where else.

Don't most of the "nigerian-prince" type scams involve some kind of back-and-forth?

Using a link to a website to start the process

I get a number of spam emails with no links. Just plain text, wanting a back and forth.

I suspect that’s more an attempt to warm up a domain/ip or something. Manually replying to emails wouldn’t scale

Hmm. I confess I don't often go into my spam folder but was just going from what I recall of the back-and-forths on https://www.419eater.com/html/letters.htm

Look up 'pig butchering' when you get a chance. It is an elaborate process of convincing your mark to fall in love with your persona and asking them to send it money. It takes months to years to execute but marks end up losing their properties, retirement accounts, life savings, everything to it.

Sure, turn loose these tools to answer the actual spammers/UCE. But:

Speaking as an ISP, if somebody turns loose what is clearly an AUTOMATED tool shitting up the contents of my abuse@ispname.com inbox with reports from some software script, I can guarantee you it goes to /dev/null

At some point we will just block their MX at the SMTP transfer point and call it a day.

98% of that already is abusive DMCA rights holders who are ignoring our federally designated DMCA-agent address for copyright violation complaints. With their automated 3rd party things complaining about people torrenting Yellowstone or whatever.

Actual reports that are clearly written by a human saying "hey it looks like this /32 of an IP address is compromised as some sort of botnet" will get a thousand times more attention. Or the very rare cases where we have a network-engineering emergency escalation and somebody calls me on the phone.

Anything generated by chatGPT or similar will be clearly obvious enough that it matches a similar pattern and comes from an automated script.

James Vitech did this, but manually and with humorous results:

https://youtu.be/4o5hSxvN_-s

https://youtu.be/IUjpoauJcKo

I think for next few years that will be cost prohibitive for 95% of Americans.

Already a reality for phone spammers/scammers: https://jollyrogertelephone.com/

There's lots of legitimate email traffic that would find itself stuck in here. I could see business questions being answered and those answers actioned on. Or legitimate sales prospecting resulting in actual orders being placed. If you choose to let a tool do your communication for you by impersonating you to the extent that another person would reasonably expect that they're talking to you, then I'm not sure you can just say "lol, that just was my spam bot" as a way of getting out of it.

It wastes my time and spammers will create new addresses automatically

Sure, but most spam will go the other way. When you're in a forum and you express an opinion that doesn't fit what the bot swarm is designed to police, they will gang up on you and argue against you from various angles, and you'll think your opinion is just super unpopular. If you are a famous person they'll do reputational attacks, after amassing karma points and followers across the swarm. Then they'll move on to combating the remaining human-powered outlets like nytimes.com, becoming the next Vox or Vice, but totally AI-powered. Their bots will spread their articles over the other ones. And finally, having driven humans underground in the dark forest of social media, they'll basically dominate the Internet.

The real sea change will come, though, when bot swarms control capital. For example, amassed for doing spam tasks. That capital can be deployed in a variety of ways, but the point is that corporations and networks will prefer bots because they generate more social capital, in coordinated ways, and also amass more financial capital than humans.

If you think this is far fetched, IT ALREADY HAPPENED on wall street and hedge funds. Bots have replaced people in trading and control the capital. Humans who trade among the bots get fleeced and don't even know it.

Letting bots make all the trading decisions is actually a good way to die due to adverse selection.

https://fortune.com/2022/06/02/zillow-6-billion-home-flippin...

> Humans who trade among the bots get fleeced and don't even know it.

Anyone who trades gets fleeced unless they know what price impact is. Don't trade, invest.

> Using these tools to mitigate spam is as likely a scenario as this, imagine that every spam mail receives a masterfully crafted response showing utter fascination and interest in SEO, or helping out a Nigerian prince. Every phone call to an unregistered number is answered by an artificial, frail, and forgetful lady that is trying her best to register gift cards.

Right. And who the fuck would pay for GPU time for that ?

Apparently phone scammers hit Americans for $40B in 2022. At this point the economic incentives are significant enough that a federal budget line item may be in order.

[1] https://www.cnbc.com/2022/11/05/how-phone-scammers-tricked-a...

The $40bn estimate was given by a company whose main product is spam prevention and call blocking.

I'd take that number with a huge grain of salt.

The comment was about using it to answer scammers, not to do the scams.

Obviously for scammers it would be huge benefit as they are directly profiting off it.

> Every phone call to an unregistered number is answered by an artificial, frail, and forgetful lady that is trying her best to register gift cards.

Kitboga has actually been playing with this. [1] Apparently his bot was successful enough to get some bank accounts from scammers that he reported.

[1] https://www.youtube.com/watch?v=maP2DwgdBts

A lot of spam is just a link to a page for someone to download malware or enter card details. It is already relatively easy to get these taken down if you care enough to, but a waste of time as the senders have moved on by then. The idea that spammers cannot make something that people tricked by their scam will use, but that insulates them from time wasters, is ridiculous.

> I’m no AI or machine learning expert so I don’t know how it works. But I am also worried that spammers could use ChatGPT to get around Gmail and Outlook’s spam filters.

This will not only increase the spam-problem, but will most likely be used to scale and do targeted phishing attack as well. I wrote an extensive article[0] where I analyzed this. And to no surprise, GPT-3 can be used to generate dynamic phishing campaigns on the fly in multiple languages, classify email responses, improve email thread hijacking attacks etc.

[0] https://www.xorlab.com/en/blog/why-ai-powered-phishing-will-...

On the other hand, ChatGPT becomes a very convenient tool for generating example data to train the ML component of the spam filter on.

something.. something... arms race

something... something... technology is not neutral.

generative... adversarial... network

Maybe the Amish were right after all.

There is a nonzero probability that a new technology will destroy the world. So at some point, the Amish will be right.

People are overestimating the importance of the message body in spam classification. The stuff that appears in your spam label on gmail is what google considered marginal, almost ham. The vast, vast majority of what they think is spam is rejected with temporary failure codes at SMTP time and never gets delivered with any label. IP reputation and other related metadata features are the key features in spam classification, and repeatedly sending different messages is not a valid test of whether the body looks spammy or not.

> and other related metadata features

Such as? Actually curious and doing a lot of sales myself, I'm interested!

Every email operator considers their classification features to be trade secrets. The closest you will get to advice from Google on this topic is https://support.google.com/mail/answer/81126?hl=en

But anyway it sounds like you intend to send spam. I recommend doing literally anything else.

The last line reads like something Clippy would say :)

Yeah, I’m starting to hear and understand that more myself. There was a extremely long twitter thread (i think from former Reddit CEO) that said the key to content moderation is moderating bad behaviour, not bad content.

I concur on this for the most part because I would say that my custom postfix + spamassassin + opendkim setup, on my self run MX, correctly classifies 75%+ of the spam or outright rejects it for SMTP transfer just based on:

a) invalid rdns of other mx

b) invalid spf

c) invalid DKIM / no DKIM signature

d) failed RBL list check - I subscribe to and feed it a few different common sense SMTP RBLs

Rejecting as spam things in the above category before it even looks at the content.

Adding a high score for invalid rdns, spf or dkim before something generally similar to spamassassin or a more advanced message subject line/body analyzing system begins classifying things help.

And then additional score is added of course for text spam content in message subject line and body.

Which RBLs do you use/suggest? My main anti spam solution is currently just Thunderbird's adaptive spam filter, but that doesn't help when I access it with a different app or via web.

Do you run your own MX? Primarily SORBS and spamhaus.

One signal for spam is just the same email going to many different people who don't like receiving it, but "the same email" doesn't have to mean it has exactly the same text.

As I have said before, the future will have two kinds of AI everywhere.

Their AI to get you to buy something, do something, believe something, or in a warzone to kill you and Your AI to protect you from Their AI. Reality may even become so dangerous and illusory that humans lose a lot of their agency to Your AI.

All this can be done since GPT3 API has been available.

I see a lot of people thinking chatgpt is something new capable of such stuff but GPT3 is far less restrictive and has been able to do all this for almost an year now.

But you must pay to use these APIs. They did give me $20 free trial though. You could make a bunch of accounts and abuse the free trial I guess. It must be cost effective for the scammers.

You can change up the prompt to change the writing style so spam filters will have trouble catching this new world of spam.

ChatGPT also throttles the number of requests an hour and has various measures to prevent bots (though not that hard to bypass, easier to pay for API)

Is it finally time for Hashcash[1]/PennyPost[2]?

[1] https://en.wikipedia.org/wiki/Hashcash

[2] https://pennypost.sourceforge.net/PennyPost

AFAIK all it takes to bypass Gmails spam filters is to resend the same scammy email that was Flagged but using a different email address. I get the same kind of scammy emails, flag them as spam, and then a couple days later, Gmail lets the same email in though it is coming from a different 123134r12345124@blahblah.com address.

In my experience the new strategy is to just send an email with no body, no title, optionally with an image of a conventionally attractive lady, in the hope people respond after which they're automatically whitelisted (as there's now a 'conversation').

I get a lot of those. Never download the image so I don’t know if there is an attractive lady or not. I do always mark them as phishing spam for gmail but it just keeps delivering more. I lost count of the free Yeti mugs I’ve missed out on.

There also seems to be a *@salesforce.com exception built in.

Anecdotally, almost every email I see in my gmail inbox is advertisement of some sort. There's newsletters I never signed up for, special offers from companies I've never had dealings with, it never ends.

Some of it hasn't even been sent to me as an email, but shows up in the inbox as though it was an email.

Granted, there's fewer scam emails than in my non-gmail inbox, but man is there a lot of spam.

If we thought the signal vs noise problem was bad, wait until 90% of data is banal AI drivel drowning out any semblance of authenticity.

Boring, brown, homogeneous noise

We’ll be forced back to the outside world

well, you don't need ChatGPT for that. I receive daily scam in my gmail with 99% the same content each time (and i mark it spam each time)... the worst part ? the server serving it is google (but the email attached to it is always a different obscure email)

I must be in the extreme minority upon reading comments on email spam, as I almost never receive a spam message in my main gmail inbox that is truly spam and not something I'm getting an email for because I lazily opted-in to a mailing list or was put on one due to association with a product. Actual, oldschool spam (Nigerian princes, claim your prize etc) literally never gets to my inbox. Maybe I'm just not a popular person. Or maybe I've kept my email private enough that it doesn't get scraped for such things? I don't understand the discrepancy between my experience and so many others here.

On the other hand it's easy to detect ChatGPT, at least for now https://huggingface.co/openai-detector

which detects the text from the article as 99% real...

I don't think message content is weighted very heavily in modern spam filters...

Things like IP reputation, sender reputation, and various SPF-like headers are far more important.

It makes sense in itself, but then why do you think the second email went through in this case, considering no factors other than content didn't change?

well this is the 2nd email from the same server, and the first was viewed by the user for 20+ seconds and not deleted, so probably not spam...

Interpretations I've seen of ChatGPT type tools, and playing around with it, I would sum up as "reducing the marginal cost of creating BS to $0". Great for content farms, spam, disinfo/propaganda campaigns.

Stuff where it doesn't have to be correct, have a high hit rate, or even be edited. Just need to produce plausible enough sounding, human-like content.

Think what can be done with an AI trained on all the data that's been collected about you.

Spear-fishing now scales.

Not all that much, considering how when you buy a blender on Amazon it recommends a second blender.

I assume most blender purchases on Amazon are for "will it blend?" YouTube channels. ;D

I think the future model of email will be spam-by-default, where you whitelist senders as you need them.

Already doing that with hey email [0] where you screen all incoming senders.

[0] https://hey.com

Considering how much crap I get in my Gmail inbox, doesn’t seem like it’s a hard task. Seriously, my Gmail account is borderline useless nowadays.

If you want to spam gmail you can just share documents with people or add events to their calendar. Apparently there's no way to stop this.

I don’t think ChatGPT is really needed here.

Gmail’s spam filters are Google’s weakest tech. At least I don’t know of anything worse.

> Gmail’s spam filters are Google’s weakest tech. At least I don’t know of anything worse.

I get a fair amount obvious spam coming through the filter, but the issue with any sort of classification is the tradeoff between False Positives and False Negatives.

The occasional False Negative causes a lot less damage (2 seconds to delete or report as spam) verses the damage of a False Positive (not seeing an important email for two weeks or ever).

I get a ton of false positives. The tech is truly lame.

This is about all its' good for, it won't even write sexy fan fiction for me

OpenAI Playground will do that. Whether it's good quality I can't say. The RLHF models are too boring and the older ones are very hard to control.

Whoever is doing this, please stop. Thank you.

only way to make email work is to let the user choose what domain they trust, and maybe to have a feature where the user must whitelist addresses.

In addition to further emphasis on "trusted senders" in the form of contact books, we'll also hopefully see a rise in identity-validated S/MIME. Though I get the feeling it'll hurt really bad before either gets deployed to a sufficient extent.

Then everyone would choose "gmail.com" and Google will get its true monopoly?

Give google half a chance and they will make "Gmail" domain only opt-out, won't even ask you to choose

over 90% of the world's smartphones are onboarding with gmail

I can’t find it but Gmail actually had some sort of whitepaper or something once about Gmail-originated SPAM and how it became a huge problem which was partially the reason they started doing phone verification.

The parent’s comment is valid. Any modern email peer is doing domain based reputation which is possible thanks to SPF and DKIM, and if you don’t have those configured you’ll have a bad time. Then it’s the job of the domain owner or email operator (postmaster) to make sure you’re not blasting out SPAM and respond to abuse feedback. If you think about it, this is the only sane way for email to function without preauthentication.

The only major outlier to this is Outlook, which is still doing IP based reputation. And of course a long tail of small server operators that rely on legacy SPAM lists from decade ago and reject only legitimate emails and pass through plenty of Viagra ads.

Reply on news.ycombinator.com