Winding down Google Sync and Less Secure Apps support | Hacker News Reader

Winding down Google Sync and Less Secure Apps support

workspaceupdates.googleblog.com

428 pointssschueller256 comments

My heart skipped a beat when I read this — I have various scripts and things that interact with Gmail.

But it's okay. App passwords will still work, it seems. This is just removal of "Less Secure Apps" support, using the account's plain account user/password.

I shudder to think how much automation would die if Google truly killed off everything but OAuth.

I don't like the complexity OAuth. I enjoyed this Perl module documentation[1] that breaks down how the dang thing works, clearly written by someone as exasperated as me (:

[1] https://metacpan.org/dist/LWP-Authen-OAuth2/view/lib/LWP/Aut...

Shouldn't be _too_ hard to convert your scripts.

I ran into the same problem and one workspace disallows App passwords. You can simply get the OAuth token with a little python script and then use it as the password: https://github.com/google/gmail-oauth2-tools/blob/master/pyt...

(see for example https://github.com/lefcha/imapfilter/issues/186)

and then have to reauthenticate it every 2 weeks because the refresh token seemingly expires for no reason...

If you set your app to "production", the refresh token works properly. Ignore the verification steps. It'll yell at you, but it'll work.

How can you set an app to production without Google verification if you're not in a workspace. I'm guessing the answer is that there is no way.

The refresh token shouldn't expire.

Maybe not for Google, but it's not part of the OAuth spec. It's perfectly valid for refresh tokens to expire.

Interesting. Many of the OAuth services I've used use non-expiring refresh tokens. Though I definitely agree; they should also have an expiry.

Me too!

I have a few scripts and dev environments that use App Passwords to send email via Gmail SMTP!

https://github.com/smallstep/cli implements some OAuth flows from the CLI, it may be helpful for you.

Thank you! This was my concern as well.

Wow thanks for clarifying. Heavy user of app passwords here.

Problem seems that you cannot set app passwords yourself and they seem laughably insecure.

16 lowercase letters (in groups of 4 separated by space), no numbers, no special characters.

Keepass evaluates this as a weak password with 65 bits of entropy (if spaces are removed)

How is this an improvement?

That should be plenty seeing as Google would lock your account before an attacker were ever able to bruteforce an app password, right?

Assuming the attacker doesn't have a leaked database of hashed passwords and therefore can't run the brute force on a local database. I want my account to be secure even if there is a leak which has been the standard for authentication for more than a decade.

This password should not be in any leaked database of hashed passwords, because it is completely randomly generated. You can't even set it.*

*unless you take this password after it's generated and start reusing it in other services, but that is pure self sabotage

It would be if Google's app password hash database was leaked.

Good point, although if that happens, then I think we probably have a much bigger problem

Is there any public knowledge of a Google password hash database leak for any of their apps ... ever?

Fair point

16 letters is ~75 bits of entopy (if they are randomly selected), not 65. The usual recommendation is 80, but it's not as bad as you say. I don't know how Keepass is doing its math, but 65 is wrong.

KeePass considers letter runs and common words to lower entropy. So “pass word pass word” fits the scheme above but demonstrates lower entropy.

The examples in their docs show that a run of characters “aaaa…” only has an estimate of 7 bits:

https://keepass.info/help/kb/pw_quality_est.html

Obviously the estimate is wrong when the password will always have a fixed length and a randomized character set. But KeePass doesn’t know that “pass word pass word” is following set rule. Perhaps parent commenter ran the calculation on an example with a run or common word within it.

You’re right it’s 75 bits for the format used by Google here.

65 bits is incredible amount of entropy. Why do you think that's not enough? I'm using 10 characters passwords of lowercase letters almost everywhere and I think that's absolutely enough.

2 ^ 65 / 20 / 365 / 86400 = 58 494 241 735

I don't think Google would allow you to bruteforce account using 58 billions of attempts per second for 20 years.

Whether 65 bits is sufficient depends on your attack scenario. I agree that Google won't allow you to try that many passwords, but for other scenarios, 65 bits might not be enough.

For example, imagine that OP is reusing passwords across different websites as most people are doing. One server gets hacked and the SHA256 password hashes get leaked, which unfortunately is still common. Currently, the best bitcoin miners can hash in the order of 10^14 hashes per second, which amounts to just 2^65 / 10^14 / 86400 ≈ 4 days of hashing. To be fair, bitcoin miners usually are not suitable for password hashing, but I'd be surprised if the NSA does not have 1000s of similar devices somewhere. Is that a realistic scenario? Probably not. But it is certainly a technical possibility.

A lower case password with 10 characters is not sufficient at all. Anycone could bruteforce that in a day with just one modern GPU.

The NSA already has full access to google’s data centers. If they want your password they’ll just sniff it in-flight.

https://www.zdnet.com/article/google-the-nsa-and-the-need-fo...

Since 2013, Google has switched to encrypting in transit across unsecured fiber: https://cloud.google.com/docs/security/encryption-in-transit...

They don't get the benefit of the doubt. I've lost count of how many times these agencies have been caught illegally spying only to apologize and keep doing it in secret. There's probably even a Wikipedia page of all the whistleblowers.

Even if they don’t have free reign over every bit of data (unlikely), there are certainly automated systems that will give them free access to any account they wish upon delivery of a rubber stamped “warrant”, signed by an anonymous “judge” in a secret court.

Don't even need that, just an emergency data request sent from a compromised municipal police email account.

1)https://krebsonsecurity.com/tag/emergency-data-request/ 2)https://news.ycombinator.com/item?id=30842757

> There's probably even a Wikipedia page of all the whistleblowers.

This list does exist.

https://en.wikipedia.org/wiki/Global_surveillance_whistleblo...

When data leak occurs, nobody's going to brute force random passwords. They'll use dictionary attacks. Using SHA-256 is strange. Some websites store clear text passwords. Some websites store bcrypt hashes. Why do you focus on SHA-256? Is there some kind of statistics that this particular kind of hash is common among hacked websites?

I agree that it depends on attack scenario. My scenario is: I expect website owners to find out about attack in a timely manner and disable all compromised accounts. Of course I won't reuse single password across different websites. Also I feel that most important websites nowadays require SMS or E-mail factor when logging in from another device, so this further decreases requirements for strong password.

And, of course, I don't expect to be targeted by government. They'll just hit my head with wrench until I unlock my iPhone, that would be cheapest attack on me, independent on password length.

> When data leak occurs, nobody's going to brute force random passwords.

People will absolutely bruteforce random passwords. There are entire communities (like hashmob net, not sure if I am allowed to link it directly) devoted to cracking as many hashes of breaches as possible. Dictionary attacks will get you most of the easy passwords, but are quickly exhausted.

> Why do you focus on SHA-256?

I chose this hash because, thanks to Bitcoin, we know how fast specialized hardware to compute that hash can be.

> Is there some kind of statistics that this particular kind of hash is common among hacked websites?

It's not the most common. That would sadly be MD5. But SHA-256 is not rare either.

> They'll just hit my head with wrench until I unlock my iPhone, that would be cheapest attack on me, independent on password length.

I agree, rubber-hose cryptanalysis can be very cost-efficient. https://en.wikipedia.org/wiki/Rubber-hose_cryptanalysis Fortunately, many governments are opposed to this kind of cryptanalysis, but YMMV.

> imagine that OP is reusing passwords

FWIW that's impossible in this context since:

> you cannot set app passwords yourself

Though more generally, password reuse is indeed a problem regardless of entropy.

Not impossible: you could get the application specific password and then start using it on other sites.

That would be foolish, but users do all sorts of foolish things.

It's not a password, it's an API Key. A GUID is 16 bytes represented as 36 characters with hypens but we consider those secure. Even Stripe's private keys are kinda short but their restricted keys are not.

> It's not a password, it's an API Key.

I don't see the difference.

> A GUID is 16 bytes represented as 36 characters with hypens but we consider those secure.

That's about twice as many bytes. That's an exponential difference in security.

They're more secure than "Welcome2024!", which is what you can expect most people to use. 16 letters is more than secure enough, you can't brute force that. They're guaranteed to be unique passwords that aren't used on other services, so credential stuffing is no longer a threat.

This is an inconsequential change to people who use password managers, but it'll help against credential stuffing attacks for the common user.

16 is a lot.

Uppercase + numbers + special characters will only give 1.3 more bits per character. 26 possibilities is 4.7 bits, so each additional lowercase letter adds enough entropy to make up for the alphabet size of 4.7/1.3=3.6 characters.

So roughly speaking, 16 lowercase letters has about the entropy of 12 characters with a larger alphabet. That seems ok to me; 12 characters is pretty decent. Certainly not laughable.

With an alphabet of 64 possible characters, you have 12lg(64)=72 bits for 12 random characters.

With an alphabet of 26 possible characters, you have 16lg(26)=75 bits for 16 random characters.

And the "random" part that allows the lg(possibilities) calculation is enabled by not allowing you to set the passwords yourself.

Of course, 75 bits only gives you about 100 billion users × apps before you start hitting birthdays—er, collisions—so hopefully they're not using the passwords as unique keys anywhere! (But why would they?)

Part of it is that app passwords are locked once used - you can't move them to a new user agent is my understanding.

> weak password with 65 bits of entropy

That's not weak?

I just hope that Gmail itself (as client) becomes something other than a Less Secure App. Right now I have to keep that LSA switch going so that my main Gmail account can SMTP send mail with credentials from my other many Gmail accounts. I've never understood why Gmail itself is a LSA in the eyes of other Gmail accounts.

I don't know why it is that, but I know why it was that at one time, quite long ago: One team had set terms for what avoids LSAness, another team hadn't updated some code to match the new terms.

> I've never understood why Gmail itself is a LSA in the eyes of other Gmail accounts.

Because Google isn't special-casing itself (which is a good thing).

You can enable 2FA on the other accounts and then use app-specific passwords, which will allow you to disable the switch.

CPAN , the jewel , the original, the king , may raku find your success

Phew. Wasn’t sure what to do with networked devices that inevitably don’t support Oauth

I had been wanting to read a document on OAuth as I have several things in pipeline where OAuth would be involved and the link you posted is just timely and perfect - so thank you!

If you can’t switch to OAuth, you can use my proxy to allow any IMAP (or POP/SMTP) client to be used with a “modern” email provider, regardless of whether the client supports OAuth 2.0 natively: https://github.com/simonrob/email-oauth2-proxy. No need for your client to know about OAuth at all.

Thank you for writing that, really helped me out back when Microsoft pushed OAuth for Office 365.

Brilliant. So this is how you could keep a client like mu/mu4e functional for Gmail and outlook365.

actually app password will keep working. Which is basically all you need for mu4e

Unfortunately this requires users to setup their own OAuth client, which is a pretty manual/cumbersome process.

Same if you're building any OAuth-enabled client from source.

(Remind me, what's stopping me from extracting the client secrets from the compiled binary, and re-using them elsewhere?)

The provider might notice that their key is being used in an unauthorized way and terminate your account, prosecute you for computer fraud and abuse, etc.

I think this is the only assured way to interop, as I expect Google may try to kill other competing apps (specially in mobile) that does not capture user data or generate data points for its ads.

I wonder if any intentional limitation here should not trigger some of the EU Digital Services Act provisions for interoperability ... in this list [1] I see Google Play, Maps, and Shopping but not GMail!

[1] https://en.wikipedia.org/wiki/Digital_Services_Act#Very_larg...

There's no good way around that unfortunately. The proxy could build in an OAuth client for the major providers, but it's unlikely that this would be trusted by default without significant effort being put into review processes.

As the readme explains, there's nothing to stop you using the existing OAuth client details from another source (such as the many already trusted open source email clients that exist).

Yes. I'd argue the problem is not on the project's side, it's on the Google side (they have ridiculously high requirements for registering OAuth clients for IMAP/SMTP use, especially for a small open-source project).

Any good guide on this (registering oauth clients) for people who want to make the move?

https://support.google.com/cloud/answer/6158849?hl=en#zippy=

There is likely a package/library for your language of choice to do a basic oauth client.

I ended up ripping the app ID out of Thunderbird and using that for my OAuth process to Gmail.

Submitted title is misleading. Turns out, it's not "all but OAuth", it's "all but OAuth and app password".

If you can't switch to OAuth, you can simply create an app password and continue using that as your IMAP password as usual.

This is surprisingly non-shitty by Google. I must admit that I didn't know that before. Can you limit such a passcode to just IMAP/SMTP, or can it be used to log in to other parts of Google?

This passcode is inherently limited to the service it bound to (IMAP or POP3), that's the whole point: don't expose your account password to something which only needs a finer-grained access.

Edit: that's incorrect, see replies.

"This App Password provides full access to your Google account"

-- Google, when you make an App Password

Ah, ok, I just tried and I'm wrong. That's correct, it provides full access.

You're directly contradicting your sibling comment. I guess I'll experiment with this in the coming days, although I'm a little worried tinkering too much will just completely lock me out of my account.

They provide full access to your entire google account, but you can create as many as you want and revoke them whenever you feel like it.

Thank you for making this. I'd been puzzling through how to build something similar. Starred and downloaded, I'll definitely be using!

It's because IMAP, SMTP and POP allow substantial access to your google account and life, yet none have the ability to do 2 factor. Nor do they allow any kind of anti-robot verification.

That means it's super easy to do credential stuffing attacks, and at google-scale, that sort of attack is going to let an attacker ruin a lot of lives.

This is a good thing, and they should have done it years ago (they kinda did by defaulting IMAP/POP/SMTP access to off - that protects most users - this is just for the rest)

I take it if someone really still want to fetch all his emails the old way, he could configure GMail to forward every single email to another address (not a GMail one): one still allowing IMAP/POP. Not a panacea but it may be an acceptable workaround for some usecases.

You can still use app-specific passwords, so your 1992 mail client will still work... Although sending even an app-specific password in cleartext over the internet seems like a bad plan.

But why are those more secure than other passwords? How can they know that this 1992 app is the app it claims to be?

Because for the user account, people use things like "hunter2". But app specific passwords are long random strings unlikely to be reused by the user for another site.

My Gmail password is also long and not reused anywhere. My impression was that it's the app itself that Google doesn't trust, in which case, why trust it with that app-specific password? Can the app-specific password still get leaked and reused if the app is compromised?

Sure but Google doesn't know that, and app passwords are a way to functionally ensure no password reuse.

What do you mean by things like "*******"? Seven asterisks?

I use Thunderbird which already had a release this year. It's been using encryption since forever.

Luckily I don't use GMail for my mail.

It's because old school passwords allow reasonable security, yet don't allow planting a user-identifiable cookie on the client machine.

You shouldn't have been voted down. I've been using IMAP, SMTP, and Pop3 for the past 30 years without a single security incident. Meanwhile, I've had many accidental lockout problems and even had to abandon accounts because of 2-factor authentication and other paranoid security measures like arbitrary "device identification" algorithms. I swear to God the goal of this security theater is only to force customers into web-based logins so contents of emails and user behavior can be tracked in an Orwellian fashion.

They said just identifiable though, and passwords are definitely user identifyable.

And most users only have a couple devices so tracking that once you already know the user is usually very easy even without a cookie.

App passwords are useless to Google as a tracking measure and are a much more secure option, especially across the entire user base.

>Nor do they allow any kind of anti-robot verification.

This is a feature.

It's trivial to add 2fa to smtp/pop/imap.

Take your preferred auth module in Dovecot and modify it to read the input password as: pwd+otp code. If the user is 2fa enabled, read last 6 digits and compare against totp.

If match, allow the IP through for x minutes or whatever other policy you want.

It works surprisingly well

Which email clients support sending the input password as: pwd+top code?

It's transparent to the client, so if the server adds support, every client gains 2FA support. The server needs to check if 2FA is enabled, and if so, try the last 6 characters of the user provided text as the OTP and the rest as the password.

It's a pretty commonly used, and works very well, but requires user education on how to fill in their combined password. A proper API with distinct fields for password and OTP is cleaner, but requires protocol support.

Isn't that horrible UI-wise? The UI will ask for password and show '******'. The user then has to remove the last 6 stars and put in the OTP.

A dedicated question for the OTP would be much better. Also, the password manager would know to not save the password+OTP every time as a new password.

> horrible UI-wise

It is, but think of why you'd build this. You own the backend and need to add 2FA support. The various client software isn't written by you so you can't change them. This approach allows the client software to add an OTP field (concat the fields for the user) but doesn't require it (user must concat OTP on password manually).

Many of the places I've seen this used don't integrate well with software password managers. OS login screen, console apps, etc; typically not web apps. But this is a good criticism.

all of them.

instead of sending your password as johndoe, you send johndoe123456 where 123456 is the code

Apple does this for older devices signing into iCloud!

This is all about getting users off of the fantastic native mail apps and onto googles own. You can’t get realtime mail notifications without gmail.app or the soon to be deprecated Google sync. I hate it. I pay Google for workspace and I hate it. At least Mimestream still works on desktop but I bet they’re coming for that soon too.

Yes, it's pretty obvious this is what's going on. The solution to these "less secure apps" was to simply generate stronger passwords for them instead of making the user jump through hoops by using oauth2 proxies, especially for older hardware/software.

Which is actually supported already with App specific Passwords...

App specific passwords will continue to work, and will work fine if your native app does not support the usual OAuth flow.

This is good to know, because I think I already use those everywhere (except on my latest machine; I should fix that).

Even so, I will also be using this as a reminder to move my stuff away from Google.

"will continue" can't really be said about anything Google with any degree of confidence.

Fully agree. Also paying for my gmail account and hate every moment of it but nobody seems to be able to get remotely close to gmail for spam handling.

After two years of using Fastmail as my primary account, in my effort to de-Google in 2022, spam recognition at Fastmail seems to me to be on a par with Gmail, if not better. I still use both.

I really have no qualms about Fastmail spam handling either. It's fine; don't let that stop anyone from switching away from Google.

Interesting, might have to give it another shot!

I use Fastmail for 5+ years now. Initially I feared that the experience would be worse than Gmail (with regards to the Web UI, to spam filtering, etc). It is not. It really is much better!

Which plan are you on?

I use the standard plan.

I love it. I can sync my Gmail to it seamlessly as I slowly transition all my accounts to my new email address.

Agreed, Fastmail is pretty good at spam filtering. My Gmail account is pretty much full of spam nowdays which seems to suggest Gmail's spam filtering isn't that great.

What makes me really mad is that I mark a sender as 'not spam' over and over again in Gmail and where does it keep going? Spam. I have a bunch of system emails and I can't convince Google to just give me my damned mail.

Strangely O365 handles most spam pretty well. I don't know why they can't do the same for outlook/hotmail/live accounts.

Hotmail is truly mind-blowing to me.

In my case ALL spam goes to inbox (a LOT) and ALL uselfull real emails go to the junk folder (very few).

I truly wonder how they do it. I can't get my head around it.

I've been gradually shifting over to protonmail over the past years and haven't seen any spam at all.

They should get on it and adopt jmap already.

It’s a good, standard protocol with unreasonably low adoption because of the chicken and egg problem between mail providers and mail applications. If Gmail supported jmap it would encourage implementers everywhere to support it. And jmap supports notifications and all the rest.

Does any service use it beside fastmail on a large scale?

I agree, but why would Google adopt it when they have their own proprietary protocol with similar functionality? Companies love anything that makes them more "sticky", whether it's good for the user or not.

IMAP does too.

No this is about getting users off fucking Outlook 2007 or worse.

Outlook 2007 users can use app passwords to maintain access. This is about users running Outlook 2007 with their main app password stored to their account database, rather than a randomly generated, email-specific password.

Any casual computer user won't be able to get this working.

If they can't get it working, they're probably not aware of the risks and should move to newer mail solutions anyway.

This only affects Workspace accounts, so I'm sure the users stuck with Outlook 2007 can call IT to get it working again.

Oauth isn't that hard to implement. It requires a WebView as part of the setup process, but after that the mail app just needs to store a token as a password and it'll keep working.

Oauth for email is part of the various RFCs pertaining to email authentication and has been around for years.

Thunderbird and various mobile apps support Gmail just fine using Oauth. I think the bigger problem is that many desktop apps seem to have stopped implementing changes to IMAP and SMTP about ten years ago.

If you have an email app that's no longer maintained, use app passwords (generated per-app passwords), like Google is indicating in the linked article. Those will still work. It's just the hardcoded username/password for the main account that's going away.

This will be a major pain for the Office 2016 users, as 30 september is still about nine months before their Outlook goes out of support, but for most users this will be quite a easy fix.

> It requires a WebView as part of the setup process

Why does it require a WebView, btw? Is there a good technical reason for that, or is it just what they happened to do?

They want the power to add arbitrary extra steps in the login flow.

For example, maybe they want to force the user to change a compromised password, or hand over their phone number, or complete a captcha, or accept a load of legalese.

You can do this without a webview in your application, but it usually means giving the user an URL to open in their own browser.

Doesn't google explicitly forbid a WebView for oauth?

The point is to not have users entering their google credentials into third-party apps, and a WebView is still entering your credentials into a third-party app. The app has to open the google login page in a real browser, not a WebView.

Oh, I see. BTW don't hardware keys (e.g. Yubikey) solve that problem entirely?

No.

Hardware keys are a second factor. But if you allow passwords to be compromised just because there is a second factor, then you're back down to one-factor auth and you've solved nothing

Fido keys can also be used without password or even an account name (but with pin which becomes the second factor in that scenario). They are very resistant to phishing because they do a challenge response every time that's bound to the domain name of the requesting site. So typosquatting tricks won't work. The private key is generated and stored in the token and they are very resistant to extraction.

In general it's way safer than a password that can be intercepted and reused by anyone who knows it.

but we're talking about google's implementation here, where fido keys are only a second factor. and in google's implementation, allowing passwords to be compromised means compromising your security, because their authentication flow is based around having more than just one factor.

Well. Hardware keys are just hardware keys. They can be used as a second factor, they can be used as a third factor, they can be used as the only factor. It's not immediately obvious that using only a password is less secure than using only a hardware key.

Don't passkey's just use the hardware key as the only factor (or at least that's how I've seen it implemented)?

I would say that if you use your fingerprint to unlock the hardware key on your smartphone, then you have two factors: one needs to both steal your smartphone (for the hardware key) and copy your fingerprint.

Or am I missing something here?

Passkey's don't strictly require 2FA sources.

Some hardware keys like yubikey's generally only prove physical presence of the key. And software implementations exist too.

I suppose it technically doesn't require a WebViee, but that's the easiest solution in most cases.

I'm not sure how you're going to implement Google's 2FA in a non-WebView solution, but if you can get the right UI and data flows to work, I'm sure you can do without.

> Why does it require a WebView, btw?

So you can't read your mail from a platform that doesn't have a web browser installed...

Edit: to be more clear: so you can't read your mail from a platform that can't run or embed a modern web browser. For example... a command line only system without a GUI.

App passwords still work. There's no need to be hyperbolic.

... for how long?

Sure you can, you just have to complete the OAuth2 login on a different device (e.g. phone) and then forward the token to the device which has no web browser.

On the phone itself the WebView limitation is worked around via deep linking (start Browser from App and once logged in start App from Browser with token).

There are tons of different ways how to do this actively being used.

What's the point of an app password?

It's a long, random password, that you generate anew for each application that needs it: one compromised app can be revoke quickly without affecting other.

But more importantly, app password can only be used for email, not other Google's service, so even if it gets leaked, the impact is severely reduced.

Your account password is too powerful, for example, it allows anyone who knows it and can provide the 2FA to change non-email settings, such as the preferred languages, the list of bank cards attached to the account, see the places visited, read and modify Google docs, or change the password. The app-specific password has a scope attached to it, and can thus be used only to do what the app needs to do, without any possibility to take your entire account over.

It's a password that allows access to email, but not to the "change my Google password" page, or to push app installation to the Play Store, or to invite other people into Google Meet links. These passwords are contained to very specific APIs surrounding email and Caldav and such.

That means you're not completely screwed when your Outlook password database gets stolen by malware.

As a user, it also allows you to revoke an application remotely without having to change your password and log in to every device again. One click of a button and new emails won't appear on a lost laptop or phone, even if they manage to bypass the screen lock. It's all about risk management.

I appreciate that we all have different bugbears, but realtime mail notification isn't going to get me off Thunderbird. Email is asynchronous, and therefore finding out 10 minutes later about a message coming in shouldn't be a problem. If I'm expecting the message then I'll be mashing refresh until it arrives, anyway.

If you need me more urgently, text me. Don't call. I don't like calls.

Same, native iOS Mail app in my case. I get Gmail notifications only when connected to power and Wi-Fi (can be set to a time interval too but I’d rather conserve my battery). When I expect a mail, I open the app and it auto refreshes until the mail comes.

My use case is OTP. Either through email or via SMS/google voice. Real time really helps with those, especially if you use Safari's OTP prefill.

I'd really rather use 1password for OTP but so few non-tech services support that.

Their realtime mail notifications are specified at https://developers.google.com/gmail/api/guides/push

You can also get realtime notifications using IMAP+IDLE+app passwords, although IIRC that'll lag by a couple of seconds.

Push requires a Pub/Sub client, which doesn't have a free tier (last time I checked).

I'm sure I misunderstand something then, so please correct me: doesn't Gmail pub to a server you choose, and you sub from it?

Are there no servers with free tiers, is it impossible to run your own, does Gmail refuse to pub to a server outside Google, or what's the problem?

I'm talking about the "Cloud Pub/Sub API", linked from the doc you posted. The pricing is described here: https://cloud.google.com/pubsub/pricing

Yeah, I am sad. I still use the Microsoft Exchange option to access my Gmail account on my iPhone in Mail.app so that I get push notifications.

> An app password is a 16-digit passcode that gives a less secure app or device permission to access your Google Account. App passwords can only be used with accounts that have 2-Step Verification turned on. [1]

Isn't it funny, how the "less secure app or device" is completely on par with OAuth-capable apps regarding security just by using a server-side mechanism Google could have promoted since… forever? Almost as if it technically isn't a feature of the app at all.

(Yeah, I get it, "apps where the secure workflow is less convenient" doesn't have the same ring to it, so the simplification is justifiable for easy communication – you will say. The greater problem is that it is kind of Google's thing to always interpret security concerns in such a way that it furthers Googles agenda and this puzzle piece is no exception.)

[1] https://support.google.com/mail/answer/185833

> just by using a server-side mechanism Google could have promoted since… forever?

Google has been pushing 2FA and App Specific Passwords for many many years. They’re just now making it mandatory for apps that can’t update to support oauth

> Isn't it funny, how the "less secure app or device" is completely on par with OAuth-capable apps regarding security just by using a server-side mechanism Google could have promoted since… forever?

They aren't on part though, Oauth is vastly less phishable and has vastly more control over specific permissions. App passwords are functionally all-or-nothing, whereas with OAuth you can configure whether something can read, modify, change settings, etc.

To be noted, the most annoying thing with Oauth2 and Google on Android is that you can't login with an email client or calendar with your Google account without your full phone to be associated with this Google account. Also giving policy right on your device by this Google account. That is totally insane in my opinion.

And you can't easily bypass that as oauth2 usage in WebView inside apps are easily restricted by Google on Android.

Can't alternative clients like k9 implement OAuth on their own? I believe I set up Thunderbird on my desktop with OAuth and it works (With office365).

Sadly the number of email clients I would trust is limited, many send off your credentials to some remote server.

EDIT: k9 already supports oauth for imap.

https://docs.k9mail.app/en/6.400/accounts/incoming_imap/

The trouble with OAuth is to get a production client ID you must pass a third party security audit.. this is in excess of $20k and AIUI must be repeated periodically. Using a developer client ID is already heavily limited, and I have no doubt now that this ladder has been pulled up, developer IDs will see further restrictions in future.

IMAP/POP passwords have long defaulted to disabled in Gmail, Gmail survived 20 years without need for these new restrictions, I can't imagine attack techniques have improved and Google's internal technical staff have regressed so substantially that they are now essential. This change seems more motivated by creating frictions for escaping the Google vortex than anything to do with security.

> I can't imagine attack techniques have improved and Google's internal technical staff have regressed so substantially

I can definitely imagine, and I in fact believe, both of those things to be true.

On a computer your are ok but not on an Android device.

To perform the oauth2 login, the app is forced to send the user to a specific Google webpage. There are only 2 ways to do that, load it inside a WebView within the app or send to an external url to be opened by the phone browser.

In both cases, Android (/Google) will capture that and use the "add account to Android" provider.

Now, let's suppose that you want to use your own custom WebView to avoid that, then the oauth2 page on Google server will perform a check against that and will refuse to load. Officially "for security reason for the user".

:-(

Those things being separate is annoying to the average person who doesn't know the distinction between Google and Android (and Chrome for that matter).

The wording is kind of ambiguous, but it seems that App-Specific Passwords will continue working

> If you have scanners or other devices using simple mail transfer protocol (SMTP) or LSAs to send emails, you’ll need to either: configure them to use OAuth, use an alternative method, or configure an App Password for use with the device.

> All Other Applications. If the app you are using does not support OAuth, you will need to switch to an app that offers OAuth or create an app password to access these apps.

The excerpt you quote doesn't sound ambiguous at all. App passwords will clearly still be supported.

The titles of both the HN post as well as the original blog post are ambiguous however.

Yeah. The HN submission title is incorrect though.

It wasn't 100% clear to me either. The wording clearly says SMTP will support app passwords, but no clear indication if IMAP will still support them.

But I asked Google Workspace Support specifically about this yesterday, and they said:

> The application passwords will support IMAP, POP, and all SMTP configurations. Rest assured that OAuth serves as an alternative, offering a straightforward configuration process for any third-party application.

The second sentence was just them continually pushing adopting OAuth. I'd love to, but the cost of the (recurring) security review is not feasible for our small SaaS app, so we'll be sticking with app passwords (thankfully).

This is just about Workspace accounts, the change already happened to normal Gmail accounts years ago.

Not entirely true.

I still access my personal Gmail account via the Microsoft Exchange option in my iPhone's Mail.app and I get push notifications this way rather than being stuck with Fetch.

The reason it still works is that connections made to Google Sync for personal accounts prior to the discontinuation date like 10 years ago are still honored and "grandfathered" if you will.

So to get it to work I just need to modify the Exchange GUID in my iPhone to what the GUID was on my phone back when I logged into Google Sync before they discontinued new logins.

Fortunately changing this GUID is possible by taking an iPhone backup, modifying a plist file in the backup, and restoring that backup to the phone.

So right now, on my iPhone 14 Pro, I am still using Mail.app with push notifications to my personal Gmail account.

I understand their announcement says app passwords will stay (for now). But should Google eventually also deprecate app passwords, it would really restrict the use of third-party clients with GMail, given the self-paid security assessment OAuth clients must undergo. It would be a sad development given that email is one of the last popular "open messaging protocols" in use, where one can choose what client to use.

Everybody is free to use something else besides GMail. Nobody forces anybody to use that.

But then there are there are large organizations who decide these things for many people.

Been dealing with Microsofts OAuth transition at work, and it's been a bit of a PITA.

Part of the problem is that it's just so opaque. You send the token and the server replies "nope", with no further explanation.

I spent literal days trying to figure out why Client Credentials-flow didn't work until I found an answer buried on their help forum saying oh yeah, client credentials flow isn't supported yet. It is now for IMAP/POP, but still not for SMTP IIRC (though OAuth is not requiredfor SMTP yet).

Next it was figuring out the right scopes, which at the time was not very well documented. Again not very helpful error messages by the servers.

Google's mail servers any better?

> Part of the problem is that it's just so opaque. You send the token and the server replies "nope", with no further explanation.

Catch-all HTTP 401s and 403s are there to thwart https://en.wikipedia.org/wiki/Oracle_attack - unfortunately, servers cannot afford to be "helpful" when it comes to unauthenticated clients.

They should have a debug mode that is user-activated for stuff like this.

I also have burned too many hours trying to get various OAuth flows working.

Yeah some development servers that didn't actually send the mail anywhere but did give usable error messages for example would be amazing.

To avoid it being used as an attack vector they could be tied to special app registrations that had to be registered with the mail development system in advance.

Microsoft is amazingly good at those opaque error messages. Such a mess of UX

I haven't been able to get `offlineimap` working with gmail in at least the last six months. If anyone has been successful, please let me know. The blog posts I've found all have out of date information. If I'm locked into their cloud, I'd at least like to have a backup of my data--

and no, I don't want the takeout service, the Maildir format and features of offlineimap are what work great!

App passwords worked for me like 3 or 4 months ago.

I use mbsync/isync instead of offlineimap, but I've had no issues using mailctl to handle the OAuth2 tokens and refresh.

Does the example on the ArchWiki[0] work for you?

[0]: https://wiki.archlinux.org/title/OfflineIMAP#OAuth2_access_t...

I will do everything in my power to pressure my university to migrate their email accounts to another service.

This is anti-competitive behavior designed to make it as hard as possible to use email client software. For example, check the procedures needed to set up oauth2 in my preferred client claws-mail.[1] That's insane.

[1] https://www.claws-mail.org/faq/index.php/Oauth2

Perhaps the headline is a bit hard to parse, but OAuth will NOT be required for email, IMAP and POP will continue to work with App Passwords

For now. They've been threatening to remove App passwords for a long time.

Thank you, where have they mentioned a possible removal of App passwords?

We use gmail at work and they warned our IT department that App passwords were deprecated and would be removed at some point (there was a previous deadline but it has been extended to a non-specific point in the future).

I see, thanks.

We had the same communications with Google that encouraged us not to use App Passwords, and hinting at a future removal. Plus the fact that App Passwords expire when the Google Account password changed, means you can’t count on the App Password working.

We are a large enough organization with that we have 25 years of use cases beyond “People Reading in their Inbox”. Printers and scanners have been mentioned. But we also have email used to automatically move data between systems.

Sometimes this is due to limitations of third party systems, sometimes we have to decide does it make sense to invest the time to rewrite a well tested tool that uses POP.

So we have decided to a standards based POP, IMAP, SMTP email service for these automated systems. It takes much less time to configure and test swapping POP server info than migrating to OAUTH

The end user email stays with Gmail.

From Google's own official documentation:

"Tip: App passwords aren’t recommended and are unnecessary in most cases."

"Tip: Don’t create an app password unless the app or device you want to connect to your account doesn’t have 'Sign in with Google.'"

"To help protect your account, we revoke your app passwords when you change your Google Account password."

"Tip: If the app offers 'Sign in with Google,' we recommend you use that feature to connect the app to your Google Account."

"If you use a non-Google app and can't sign in, the app's sign-in process might not be secure. Try to update to the latest version of the app and use 'Sign in with Google,' if it’s an option."

Yeah, no problem at all. I'm sure it's going to work flawlessly and effortlessly.

My SO uses GMail and anything but Thunderbird it's a hellish nightmare to setup. On the HN comments about "but my secUrity"... who cares. Get a good and long password and use TLS everywhere. This is enshittification and pure EEE tactics against public standards and free sofware email clients and yet these careless users applaud it.

The nice thing with public standards is that you do not have to patronize providers who do not comply with the standards.

> This is anti-competitive behavior designed to make it as hard as possible to use email client software

Use App password. They are not blocking any 3rd party clients. Are you advocating using real password in every random client that may or may not transmit it in full text elsewhere?

I do assume you know security essentials.

I will do that if our university decides to switch on 2-factor authentication (which would be total chaos), but my trust in Google's ability to make this effortless and easy in third-party email clients is zero. See my other post why.

Regarding security: Google had physical fiber optics connections to the NSA and claimed they didn't know about them. It's not a very credible claim but if if was true, then it would be proof that Google has no competence in security at all.

Any decent 3rd party client does this already. Take for example, thunderbird or k9-mail - major open source ones. Even Mail (from Apple, macOS) does this. What else you need? Sure if you use mutt or ALPINE read the related forums for help.

while anyone can criticise any large company one should do it for the correct reasons.

Your frankly-speaking somewhat condescending reply fails to address the issue. This has nothing to do with the client software, which is claws-mail in this case. My organization does not enable 2-factor authentication. Once they do, claws-mail can be configured to use "App passwords."

But if you look at my other post, quotes of Google's own documentation of "App passwords" and various testimony in this thread do not inspire much confidence that this will work as an acceptable long-term solution. As I said elsewhere, the rationale behind all of this seems to be anti-competitive behavior. It's not new idea to disguise anti-competitive behavior as security issues (cf. e.g. Apple's code signing and Gatekeeper). This also explains the misleading and incorrect term "Less Secure Apps" Google uses.

Happy to help onboard your university!

We currently serve UMD, Tufts, Swarthmore, and more.

https://forwardemail.net

I read through your website and I don't understand what your service is. You sorta seem like an email host but seem to very intentionally and carefully not advertise as an email host.

It's shameful how far we've gone down this path. My university is using Microsoft Exchange and they refuse to allow IMAP access even with Oauth2, only allowing you to use proprietary protocols. So far I've only seen Evolution and Thunderbird working (with a paid plugin for the latter). I think it's a means to exert control on your email usage. For example, most sane clients do not show images by default, which prevents various types of tracking pixels from working, whereas Outlook doesn't do this afaik.

Here's updated info on Outlook:

"Microsoft Outlook is configured by default to block automatic picture downloads from the Internet. You can, however, unblock pictures that you think are safe to download."

From https://support.microsoft.com/en-us/office/block-or-unblock-...

Why is there AFAIK no tutorial on how to steal the "app token" (sorry, I don't know the proper term) from Thunderbird or other mail clients?

I don't think the title is correct. IMAP/SMTP will still be supported with App Passwords. That's my reading of it.

They really buried the lede here. Google Sync is their implementation of Exchange ActiveSync. Google Sync (or IMAP) is the only way to send emails with aliases on iOS with Apple Mail.

This is terrible.

People giving away their google account password to other apps / companies is bad for security. I'm surprised this still existed.

I think this hasn't worked for accounts that have had 2FA for quite some time. I remember having to switch several years ago.

I gave Google my IMAP account from another Google account in order not to have to check two accounts and have the mail forwarded...

Any decision that limits my ability to use third-party apps, accepting, etc. is a compromise not worth making.

It sounds like there is a feature for generating additional passwords for specific apps/scopes?

Anyway. Forced 2fa without warning has locked me out of several Gmail accounts. I didn't know when migrating took Proton that you must use their client for mobile.

All these violations of abstraction!

Yep, this is the reason OAuth2 exists.

It's not always bad security.

a) I have apps with their own Google accounts

b) I have devices with their own Google accounts

It is (or was) a good way to go. Receiving an email from a scanner or sharing something with an app on Google Drive is a lot more secure than OAuth + "This app will have access to EVERYTHING IN YOUR GOOGLE DRIVE"

At a certain point, you'll have apps and devices that have (or have no) permission to use someone else's account that was previously under your control.

> "This app will have access to EVERYTHING IN YOUR GOOGLE DRIVE"

Then don't use such app

I don't think you get it. If you want to automate anything in Google drive, that's the modularity of permissions in Google APIs:

https://developers.google.com/drive/api/guides/api-specific-...

I automate things all the time. If you want ChatGPT to provide feedback on a document, the above is what I've got to work with. If I want a web app to maintain a spreadsheet, again, that's what I've got.

Granular permissions = more secure.

I'm not giving away anything to anyone when I locally install and configure an email client on my computer to access my gmail email account. It's my software I control on my computer.

The idea that people should only use a google application to access google email sounds crazy to me but I understand the situation is different on smartphones where you aren't in control.

You have to trust the email client's developers to not be malicous, to not write insecure software, to not get hacked, and not sell to someone malicous. And on desktop it's worse since they are less secure as programs can typically read each other's files meaning some random program can read your Google account password that the email client is using.

I don’t mind two step authentication using TOTP but as soon as you sign in to an android device with a google account, google decides to use that device for two step authentication and there’s no way to stop that short of signing out of google on the device.

But also how do app specific passwords protect you if you have malicious software on your computer rifling through your files?

>But also how do app specific passwords protect you if you have malicious software on your computer rifling through your files?

It minimizes the blast radius if it is compromised. A Google account provides access to much more than just email.

App-specific passwords are limited to just a couple of services, so somebody stealing one of them can cause a lot less damage than if they got the actual Google password. The app-specific passwords are going to be unique rather than something you've reused on dozens of services, so the password being stolen won't be automatically pivoted to compromising your other accounts. Finally, their use can be audited, and each app-specific password can be revoked independently of each other and of the credentials giving full access to the account.

It's the very same with trusting Google, and my trust in Google is much, much lower than my trust in the developers of the applications I use. Google is a fairly untrustworthy company, which is why I don't use Gmail personally. Unfortunately, I'm forced to use it at my university.

Libre software it's a thing. Is not 1990 any more. Also, passwords in mail clients are often encrypted.

>Libre software it's a thing.

Most consumers do not know how to check source code for backdoors. Most consumers do not know how to compile from source.

>Also, passwords in mail clients are often encrypted.

Which means they have to be decrypted at some point. Malicous software can decrypt it itself, or steal it after it has been decrypted.

If I believed in conspiracy theories, I'd say that Google encourages the security theater* industry to make you distrust your devices so they can have all your data.

* There are real security vulnerabilities, and there are end-of-the world articles that try to make you believe the whole world is at risk via some complex exploit that requires the attacker to obtain local root some other way.

There is e-mail, and there is Gmail.

(By the way, there are a lot of great e-mail providers other than Google. Often even with a web UI superior to the one by Gmail.)

Example: Fastmail.

Changes to Gmail syncing with other email clients

https://news.ycombinator.com/item?id=38975692

It's important to note here that OAuth isn't one thing anymore like it was for OAuth 1. With OAuth2 it's more like OAuth is a toolset for implementing your own proprietary protocol. Each major provider needs it's own custom implementation in email clients to cover it. There's no general purpose oauth2 in practice. OAuth2 is more like a per-corporation protocol.

Lets hope they don't sunset "app passwords" for at least a few more years. If you haven't yet, consider setting up your own mailserver for kicks and using it for non-important things to build IP reputation for when public email stops being email entirely.

Google isn’t really worried about password entropy beyond a reasonable amount. The primary threat model is phishing. This is why multifactor is so important and once once you have that enabled, nobody gives a shit if you even rotate your password. Just needs to be long enough and not guessable because it’s not the sole means of authentication.

Probably not a good idea to have something as critical as one’s primary email account identity tied to only a single factor of phishable credentials.

Requiring App passwords seems better, but it bypasses requiring a MF.

oAuth, while a a beast, seems even better as the workflow still initially requires a second factor.

I host my own email server, and I want to tell that with a static IP with reverse dns, spf, dkim and dmarc I had zero issue delivering mails. It might sound complicated, but it is not that hard.

True! Also you can pay 2 €/month to Hetzner/$webhosterofchoice and have them handle that for you... No need for Google after all is what I want to say.

You know what would be nice...

Google going through their access logs and sending every admin an email list of usernames and services impacted by this change.

Ie.

The following users in your organisation are logging in with password-based "Less Secure authentication", and will need to migrate to another login method by 30th September:

[list]

Logins were to the following services:

[list]

For a full breakdown of which users logged in to which services insecurely on what date/time, reply to this email and we will auto-respond with complete logs.

For help resolving this issue, see [our help article].

They did, I got an email, with an attachment spread sheet containing users email addresses on my domain. Bottom of email stated "A list of affected users is attached."

me too? The file was named users.xls.exe and it just opened a weird black screen for a minute and closed. idk

I got an email listing my users, luckily only 4. What i'm trying to find now is how to run my own report so i can see when those users are 'fixed'. Anyone able to point me in the right direction? Thanks

you probably can't - the report is probably a massive mapreduce over all of googles access logs - the kind of thing they can do once, but they won't add a button to the dashboard for you to do yourself.

Thanks. I've spent the morning looking and that would explain why I couldn't find it.

So if I am connecting to my Gmail on my iPhone via the "Microsoft Exchange" option and using an app password I will still be OK?

The reason I still connect via the "Microsoft Exchange" option instead of picking the "Google" option in Apple's Mail.app is because I get Push notification for my Gmail when using the "Microsoft Exchange" option compared to only having the Fetch option available for the "Google" option.

After reading more carefully, I guess my method is done for.

"As part of this change, Google Sync will also be sunsetted: Beginning June 15, 2024: New users will not be able to connect to Google Workspace via Google Sync."

"September 30, 2024: Existing Google Sync users will not be able to connect to Google Workspace. Here is how you can transition your organization off Google Sync. To find Google Sync usage in your organization, please go to the Admin Console, navigate to Devices > Mobile & Endpoints > Devices, and filter by Type: Google Sync."

That really sucks, I prefer the Mail.app to the third party Gmail app and I liked push notification e-mail compared to Fetch on a 15 minute timer. Why can't google do Push notifications through Mail.app yet?

The limitations and clearly second-tiering of the Exchange connector are what pushed me to move away from Google Workspaces last year. I settled on Fastmail, precisely because they support Push in iOS Mail.app. From my research, they are one of the very few who support the imap extensions required to do that. Have been very happy with them so far, they support all my workflows, and do so a lot better than Gmail.

Note that Google has a very complicated process to migrate an IMAP/SMTP app to OAuth. There is a long list of requirements to register an OAuth client for these protocols: https://developers.google.com/terms/api-services-user-data-p...

It would be great if Google supported rfc8414 and rfc7591. Right now most MUAs hardcode credentials instead of auto-discovering/registering/configuring them and decline to implement those standards "because the big boys don't support them". The practical result is that one cannot use oauth2 on their domain easily: the MUA needs to be told about which set of oAuth2 creds to use.

See https://searchfox.org/comm-central/source/mailnews/base/src/... , https://github.com/thunderbird/autoconfig/tree/master/ispdb and https://bugzilla.mozilla.org/show_bug.cgi?id=1602166

I wonder if this was triggered by the 23andMe breach?

In that, it became pretty clear that the press and the public will blame a company if user accounts are breached even if the user had a weak/reused password.

Since then, companies kinda have a responsibility to protect the users data despite the user doing stupid stuff.

No. This was originally announced in 2019 and has been proceeding in phases since then.

About 10 years ago we set up a pretty slick integration from our office wifi controller to Google's account directory using WebDAV and Radius. It allowed us to have a WAP for insiders who could authenticate with their Google-issued one-time-passwords (needed because normally a Google login requires MFA if configured) and access our internal network with their individual Google accounts (no sharing of wifi credentials; no separate wifi login to manage for offboarding).

It was certainly Less Secure by modern standards, but it saved a boatload of time for everyone to be right on our internal network as soon as they connected to wifi instead of having to VPN in.

The problem with gmail's OAuth is the initial setup. After that is done, it works fine for years. First, you need to create an authorization token and a refresh token. If your authorization fails, you need to use your refresh token to get a new key. But the most damning part is: The initial process which gets you a refresh token requires you to login via browser. That's hard to deploy on a headless server. At some point I remember having to create a large enough swap space to run UI, install xorg, install google-chrome, go through the graphical process, then uninstall xorg and remove the unused swap.

Man, the way Google treats paying customers is appalling... It seems very frequent that they remove [WIDELY USED FEATURE] on a whim and tell users to pound sand.

Does anybody know what makes refresh key is more secure than login/password? I'm quite dumb and don't understand how OAuth make apps more secure.

Also maybe someone knows how to get read only token for IMAP access? Last time I've checked there were no any scopes. One literally could use same token to access and IMAP and SMTP. But IMAP/SMTP token split could really improve security.

because apps do not get credentials, so the token can be easily revoked just for this app. thats why app passwords are still fine.

Thank you, it makes a lot of sense.

News from September (2023)

Email was sent out today to many Google Workspace admins with a link to this blog post.

If I'm accessing IMAP with an app password will I still be able to do this after the Sep 30 deadline? I've re-read this 3 times and I have no idea. It almost sounds like they are disabling anything other that XOAUTH auth method from IMAP (which, I think, they already did for non Workspace accounts).

Made the swtich to Tuta (previously Tutanota) about 2 years ago and never once looked back. I only wish they had support for some sort of protocol for third-party clients.

That was a turn off for me personally. While I think more expensive per user (but free for up to 150 emails per day without your own domain name), ProtonMail has "Proton Bridge" that works wonderfully for me on Windows 7/11. It gives you what the name suggest, its a bridge between your cloud encrypted emails and sets up local SMTP that you can connect any program to (I have Thunderbird and Outlook). Haven't had any issues at all.

Is Proton Bridge free now? I remember it being paid.

This rollout plan seems misguided...

You're going to remove the option from the admin console before sunsetting the feature? So users can't turn it off to test/see impact?

@dang the title is misleading as the user can also "configure an App Password for use"

I'm accessing my Google Drive through rclone, I should check if this affects it.

It won't. rclone uses OAuth.

Oh, cool. Thank you.

What's the way to go with backends that send emails using gsuite's SMTP?

Flagged, because the title is both editorialized and factually wrong.

And I migrated away all my accounts from Google in anticipation of it being more and more user hostile.

Honestly, if I were still at Google, I would be outraged. In the name of false security, they make the wall around their garden taller and thicker every single day.

Is this going to be a problem for my mutt and mbsync setup?

Oh no how will workplace printers scan to email now

You can still configure an app password

Good luck with that. I couldn't get it to work and to switch to a mail account on another non Google host.

Already before the LSA would disable and you would need to go into the settings to re-enable it all the time.

It’ll end up as a local POP or SMTP daemon which redirects email through an OAuth’d app.

Everything old is new again, we’re running services locally just to make legacy work.

App Passwords have always worked fine for us, and have done for years!

How do they do it now?

The one at my work sends the PDF as an attachment, from its own email address.

I use smtp2go for that, it's dead easy.

Welp. Add Google Sync to the Google Graveyard

Reply on news.ycombinator.com