Oauth2 support for GMail
pmail.com
pmail.com
By the looks of it Pegasus falls into this category and should not have any issues getting approved (still need the YT video and such but the Google team are surprisingly responsive and helpful in my experience).
Do you have evidence - in writing - to the contrary from a Google official?
Abridged wording and my non-lawyer interpretation below in case I'm not clear:
> Every app that [accesses Gmail and also accesses other servers] is required to go through a security assessment from Google empanelled security assessors. [...]
> In order to maintain access to restricted scopes, the app will need to undergo this security assessment on an annual basis, [... costs usually] between $10,000 - $75,000 (or more) [...]
> This fee may be required whether or not your app passes the assessment and will be payable by the developer."
For all my fear mongering, I should point out that the only reason Google are saying this is to cover their backs when they decide to levy the maximum fee on an unsuspecting competitor. If they don't consider you a direct competitor, you might be ok. They have no reason to use this policy to alienate the majority of desktop applications that connect to email.
But they also have no repercussions if they do.
They don't just send you a bill for $10k. You can opt-out of the yearly audit by removing your use of restricted scopes.
But yes, it is a yearly required audit, and they're serious about it. This limits the kind of apps that can be built on Gmail (basically – no free apps), but it is undoubtedly better for end users.
Having gone through the process, which checks among other things that data can't be resold, tokens are encrypted, user data is really deleted when you say it is, and that Gmail API access is auditable in the event of a breach; these are all good things for users.
(My auditor was so thorough they actually found a high-impact XSS bug in Firefox – the bounty covered part of their fees.)
A new compatitor has implemented a feature where they use the Gmail API to analyse a user's email exchange (from their servers) in order to recommend who to opt out from. It's very effective, and even though I would never give a 3rd party access to my email account most people do.
Now we thought of implementing a similar feature, but from the client side (which is a bit less effective but much more privacy respecting). Now I'm not sure from what I read if we need a security audit or not, but the risk and the extra work isn't worth it for us. We're a nonprofit.
Our compatitors on the other hand are a VC backed commercial organization. They make money buy providing a service to the companies they help people opt out of. The whole opt out side is just a way of manufacturing demand, so you can guess where their loyalty lies. But because they are well funded and because sending opt out emails is basically marketing for them, it makes sense for them to pay for an audit.
At the end of the day, the user suffers.
and I've added the quotes because of course everyone else does that, and it trains users (and now since everyone is a user it basically conditions society) to have unrealistic expectations, skews what people value, completely fuck up the market (hard to compete with free)
...
that said, in the end the user gets what it wants "opt out", and it's free for them.
Software that, for now, under Google's current interpretation of the rules, is allowed to use their OAuth without paying these fees.
The question I'm asking is, what happens next year when Google decides to silently change their interpretation of the rules? Do you, as a FOSS email client writer working on JohnnyMail, risk a massive yearly bill of 1/6th or more of your salary that you are contractually obliged to pay - or just say "Sorry Google, you've outpriced me" while their interpretations are still favourable?
It's not "undoubtedly better for end users" that free email apps be excluded from Gmail. It's not better for end users that open source software developers are given a sword of Damocles hovering above their heads. Sure, it's undoubtedly better if these free apps can be guaranteed to be secure, it would be even better if Google could do that in a way that didn't cost a massive amount or a surprise bill.
I'm glad you had the resources to be able to go through the process, and that you found it a useful process to go through. But it doesn't justify the uncertainty.
I suppose Google could charge for future access. Any platform could. But not retroactively. That would need to be in a contract and it’s not.
Yes, developers and business claim they make user data, privacy and security a top priority. As we have seen from plenty of developers on the facebook platform, if not checked, they far to often lie, betray users trust or are just totally incompetent.
At least on the business side, giving restricted scopes access (ie, enabling a third party server to read all emails in a domain) is a major permission. It needs to be treated like this. In many cases a problem here unlocks a LOT more because email is used the default password for everything (via password reset options and more).
I hope google holds a firm line here and doesn't bow to hacker news type social media pressures - we have too much evidence of bad and poor behavior by developers to just trust them.
I can tell you that for businesses and others spending money (ie, where the business is the customer and not the product) the perspective is opposite this.
A business wants google to track users so logins from unusual locations / devices go through more rigorous authentication. That is considered a benefit, not a harm.
A business wants google to scan everyone's email - for everything from phising to spam to malware. This is considered a benefit not a harm.
I think folks here underestimate just how trusted and core to many individuals and businesses google is. Many folks trust google MORE than they do their own goverment, including on issues of spying on emails and more.
The goverment leaks everything - from photos of dead celebrities to tax returns. Many goverment are active in spying on their users as much as they are able. Around the world, brands like Apple and Google considered evil here on HN, have just insane brand value.
Again, Google are idiots if they were to go the facebook route and not keep the private info they hold pretty secured. The downsides are SO much larger for them (see Cambridge Analytics) than the upsides of allowing random third party internet developers to access someone's email on an ongoing and programmatic way without these types of controls.
The point isn't that Google is evil and random devs are good. The point is that Google is amoral and harmful - and also enormous and powerful. Random small developers may be good, bad, whatever, but they are diverse and individually powerless. Should you be more afraid of the Stasi or of a neighborhood burglar?
> Every app that requests access to restricted scope Google user’s data and has the ability to access data from or through a third party server is required to go through a security assessment
An email client that only transmits data to/from Google's own IMAP/SMTP servers does not have the ability to access data through any third party server, and thus does not require the audit.
Source: https://support.google.com/cloud/answer/9110914?hl=en#zippy=...
(I'm presuming the word "accessing" is used here to mean any use of a third party server, regardless of whether read or write - because the whole idea is pointless if transmitting is not included in the definition)
It also includes any email client with a built-in VPN, or potentially any client that can use a VPN (remember, it's at Google's discretion)
> [accesses Gmail and also accesses other servers]
... because that's all that convoluted line means. Break it down:
- Access to "restricted scope Google user's data" (in this case, all we care about is Gmail)
- AND ability to access data from or through a third party server.
It's that last bit that people seem to be getting confused about. For example:
- if your app accesses Gmail and Hotmail accounts, then your app is doing both
- if your app accesses Gmail and also checks today's weather, you're doing both
- if your app accesses Gmail and sends basic usage telemetry. Or checks for updates. Or has plugins that provide spam checking or virus scanning... you're probably doing both
- if your app has ANY plugin system, it could be argued that your app is doing both.
While the language may be unclear, "third party server" is probably intended to reference any non-google service.
And my overall point still stands: YOU do not get to decide what triggers their security review. All you have the right to do is pay the bill.
It might not be required for applications that run locally, but they don't tell you whether or not it will be required until after you've already done the work to create the app.
The exact wording from the FAQ is:
"Local Data Storage: Local client applications don't need to undergo a security assessment because data is run, stored, and processed only on the user's device. Local client applications that only allow user- configured transmissions of Restricted Scope data from the device may be exempt from this requirement."
Keep in mind that any email client that allows you to reply to (or forward) an email would count as transmitting restricted scope data from the user's device.
Not if it does so only via the oauth api?
> Ensure your app complies with the Google APIs Terms of Service, Google's API Services User Data Policy, and the Additional Requirements for Specific Scopes, which includes undergoing an annual security assessment if your app accesses restricted scope Google users data from or through a third-party server.
In the case of an email client data is transmitted directly from/to Google's own IMAP/SMTP servers and not a third party, and is thus exempt from the assessment.
Is there any reason it can't work on Windows 7?
So Pegasus Mail accesses your email from their servers? For reference, they have an entire flow designed so that the auth credentials never touch the app developer's server for desktop and mobile apps - https://developers.google.com/identity/protocols/oauth2/nati...
It's easier to rule out undetectable-by-google data exfilteration if the app can only connect to Google.
The obvious way around this is to make a Google-only edition.
Yuck.
In fact, in Google's guidance on this subject, they say:
> Local client applications that only allow user-configured transmissions of Restricted Scope data from the device may be exempt from this requirement [to get a Letter of Assessment].
And in another FAQ:
> Local Data Storage: Local client applications don't need to undergo a security assessment because data is run, stored, and processed only on the user's device. Local client applications that only allow user-configured transmissions of Restricted Scope data from the device may be exempt from this requirement.
My feeling is that the author of Pegasus Mail has checked a checkbox incorrectly somewhere, or alternatively has not implemented the desktop oauth2 flow correctly.
My unfounded guess is that just like the parent here, OP isn’t very knowledgeable about OAuth and chose the wrong flow.
That really doesn't help OP as their customers most likely won't be capable to run that.
More details on the developer:
https://en.wikipedia.org/wiki/David_Harris_(software_develop...
When Google disables imap at the end of the month for gmail (only allowing OAuth2+imap, which is not imap) that's the end of gmail for me.
I don't think many email clients support OAuth2 (Thunderbird does, but that could be Gmail specific?) but the concept isn't inherently bad, in my opinion.
I don't really see the problem with OAuth2 itself, the email space is just very very slow at accepting new standards. Dovecot and UTF8 email addresses still don't play well together, for example. The lack of proper on-the-fly application registration for mail servers is annoying, but supporting them should be easy enough if the mail service isn't run by complete doofuses.
To be clear, I never said OAuth2 was google only. I said the standard OAuth2 is so much not a standard that an implementation written for Google's imap+OAuth2 will not work for any other email provider's imap+OAuth2.
No, it definitely works on Office 365 too.
OIDC builds on OAuth2 to fill in those gaps with prescribed details and add additional systems necessary for a workable federated auth system. it _should_ be possible to use it entirely provider-agnostic fashion, but in practice it's like any complex protocol with optional parts in that implementations have incorrect bits or idiosyncrasies (why does Azure's provider sometimes serve public keys it doesn't actually use to sign tokens in some modes? god only knows) and you'll probably only provide integrations for a subset of providers, not just let anyone with a discovery link show up and use it.
I mean if you take some email client application that went out of it's way to support OAuth2+imap for google via a plugin then even if you replace all of google's info in the plugin it won't work for other OAuth2+imap implementations. Every OAuth2 implementation is a different thing, as you say. But the megacorps are pretending it's some standard protocol. It's not and the fallout will be much worse than people having issues because they picked easy passwords.
This is massively different than actual protocols like pop3 or imap where all you do is change the info but the protocol stays the same.
As a gmail user, this actually seems like a sensible thing. Gone are the days when users are trusted to just click 'allow' to let some random third party access all your data. It turns out, users don't fully understand the ramifications of that, and complain loudly when the 'free personality quiz' they allowed goes and downloads every private email to use for marketing.
So, if I can't give access to my mail to a random untrusted third party, the next best thing seems to be to give access to someone who has at least had their systems audited and who at least has enough money to pay for an audit, meaning they probably have some business plan and can be tracked down by the courts if they mishandle the data.
Which is why I never published it (although I have been using it for almost two years), and setting it up requires a series of laborious steps of activating the Google Developer Console, creating API keys etc., which I found just to embarrassing to put in a README :)
this is my main gripe with oauth for login -- the amount of overhead for getting it working is absurd. it's not private (goog knows which users use which sites) and there's too much preapproval (company consuming oauth has to have a dedicated google project for each login system)
better, simpler system would be 1) client site issues random token, 2) user forwards token to google, 3) google signs bundle with user's email address + client token, 4) client site verifies bundle with goog's public key
all communication should be through the client, the three way handshake is absurd and some legit sites don't implement the standard correctly
oauth for access federation maybe should require more setup but idk -- I feel like we haven't thought through access federation as a society
The cost of the assessment typically varies between $10,000 -$75,000 (or more) depending on the size and complexity of the application
As he says, even if Google classed his as a small app (unlikely)
"Regrettably, these kinds of fees are far beyond what I can afford, given that I rely on donations from my users to make ends meet: even the $4,500 fee is well beyond what I could find on an annual basis. "
I get that doing proper auditing is expensive and the costs are within industry rates but to me it reeks of slowing turning the screws by boiling the frog on the slow path to monetization. (many mixed metaphors I know)
What?!
They don't care if the videos are terrible quality, it's only for them to understand what they're approving, and to have some documentation if a developer later changes an app to do something nefarious.
I'm currently using https://mimestream.com/ and I somehow doubt all of them had to pay that kind of upfront money.
> The cost of the assessment typically varies between $10,000 -$75,000 (or more) depending on the size and complexity of the application; smaller applications may see costs at a lower threshold of $4,500
That's just bad, but pretty typically google.
For example, my email is in Google. There are things related to e.g. evidence for litigation from many years back. I have documents shared with me in Google Docs.
Google used to be pretty good about "don't do evil." I've degooglified what I can, but I can't degooglify 100%.
I guess what I should do is stop putting new stuff in Google.
> Tip: App Passwords aren’t recommended and are unnecessary in most cases. To help keep your account secure, use "Sign in with Google" to connect apps to your Google Account.
So, for Google, App Passwords are clearly "second class citizens", deemed insecure and not recommended. And, as an app developer, you probably don't want to be seen as recommending an insecure authentication method?
Really the biggest downside about the OAuth flow is that it requires a relationship with Google (a client key). If this could be a fully decentralized standard it would be fantastic.
(Google Takeout is a huge pain and not at all live. I guess they see it as their data, their rules..)
If not, make a clear cut, purchase domain, and never ever be at the mercy of mail provider.
They likely figure most will choose option#2 since it's so painful/expensive to do #1.
After submitting it took ages to get reviewed and even when it had been processed the status wasn't entirely clear.
At the time it was made clear that using app passwords was considered a risk and users were receiving emails about it being a security risk.
* Asking for huge amount of money in the of the process (after you wasted hundreds of hours of expensive developer time) * forcing to use restricted scope (read,White,delete) for standard IMAP access even if you need only read-only access or use their proprietary API with read-only scope with unknown rate limits and which is difficult and time-consuming to use * cryptic messages why app is not approved and ignoring arguments made for their questions * often changing APIs (access token formats)
They do everything to avoid competition not making better product but by locking existing customers to their ecosystem.
And this is not only google. We had similar issues Apple and Meta.
Like I said, there are things I don't love about the program. But it's not an elaborate hazing ritual.
Are there any publicly known examples of this? I'm not doubting that it's happened, I've just never actually heard about any cases of this with respect to the Gmail OAuth API specifically.
Read only access to the user’s inbox is enough to reset every password so I’m unsurprised that they make this hard.
>...[an] app that requests access to restricted scope Google user´s data and has the ability to access data from or through a third party server
Gmail and other huge centralized points of censorship and surveillance must be destroyed. If you are a user, move away. If you are a developer, do not support these closed systems.
This is not some principled stance, it is a clear and practical command: stop using these services.
If your personal email address ends in @gmail.com, you are doing it wrong.
This is a super common misapprehension people have about data locality and surveillance, and it's no surprise that marketing plays off it ("keep your mail in Switzerland, so it's subject to Switzerland's strict privacy laws!").
There are other reasons to house things in Switzerland (or the Lesser Antilles or wherever). I'm not saying you should use GMail (unless security is a concern of yours, in which case you should use GMail).
This is no longer true, as Ed Snowden showed us. This is literally the point of their secret interpretation of FAA702. Pretending otherwise is nonsensical. The USG, just like the CCP, gets whatever data they want, about anyone, from servers in their own country, without due process.
Anyway, my comment was not about jurisdiction, just about surveillance.
Regardless, Google obviously has the plaintext if you use Gmail. You don't want that, either. The USG will probably only use it for a subset of things; Google has no/few limitations on the use once you voluntarily give them such a huge trove of data about your life, travel, vendors, and purchases.
There's also the issue where Google itself is mass surveillance.
Secret services swap their data, so they can all read my santa letters.
There is the viewpoint that one should prefer one's own jurisdiction regarding storing data (I know you don't like Schneier but he e.g. holds that viewpoint).
I use an email provider from my country. If a judge from my country wants to persecute me, they have to complete a legal request which is tested by my provider.
If I used an email provider from Switzerland or the US, is that also the case?
Also, it's important to think of the economical aspect. If i financially support providers in my home country that lobby for better privacy laws and don't engage in surveillance capitalism like Google does, isn't that creating a good alternative that can also be properly regulated?
Plus, it's one dependency less from a (more than) half MAGA country.
So your recommendation makes sense if you are a citizen of the USA, but not everyone here is.
Addendum: what do you actually mean by "security"? For me, that is also "being able to access my email account whenever I want" and that this account doesn't get closed, even if I do shit on the internet, whoever that decides. Google shouldn't have the right to say "these actions are uncool so let's kick this guy" (never happened to me, but I don't even have a contract with Google!, they have the right to cancel my account whenever they want.).
I guess I am so naive...
https://en.m.wikipedia.org/wiki/Antitrust_cases_against_Goog...
- in terms of privacy, applications that have access to your Gmail inbox now require a security audit.
- the audit is not required for MVP (<100 users) or applications internal to a Google Workspace org.
Of course, you have to pay for the audit. But:
- it’s only required when you ask for restricted user data (i.e. reading my emails).
- Google doesn’t take 30% of your revenue to use its API - which is free by the way.
To me, Google has created the perfect world for developers here. And that says a lot when I read the developer of Pegasus Mail doesn’t want to record a 2min long YouTube screencast to get approved.
Also, just wanted to kudos the Google OAuth team that has greatly improved their process in the past years. If you follow the guidelines, you can get approved within a day now.
Your criticism was completely unnecessary. It doesn’t say anything about the Pegasus dev except that he’s normal.
How is that in any way related to your comment that he shouldn't complain about it? If it wasn't a requirement then there would be no reason to complain in the first place.
And that's not even what his main complaint, his main complaint is that he shouldn't have had to record the video in the first place because the warning about this feature costing money should have been documented somewhere together with all other requirements.
Basically Google is behaving like that website that tells you "your password should be more than 8 characters", change password submit, "your password should be less than 16 chararcters", change password submit, "should contain at least one upper character", change password submit, "should contain a special character", change password submit, "should not contain single quotes or other characters that are too special", repeat ad nauseam. If you have requirements document them up-front and all in one place.
This is after jumping through a bunch of hoops to create a GCP app. It even complains about the pseudo-app being from an 'Unverified developer' despite that developer being me.
Congrats on misreading the article. That's not the problem. The problem is having to spend a lot of money for a security audit for free software.
100 lifetime signups isn't anywhere near enough to know whether or not an app is commercially viable. If the cap were something like 10,000, or even 2,500, then there would be a lot less complaints.