Gmail’s API lockdown will kill some third-party app access, starting July 15
arstechnica.com
arstechnica.com
1. It makes sense that Google wants to stop apps from abusing their storage platform. There are a lot of projects that abuse the data storage capacity. There was that one app that converted files to Base64 or something and was storing files that way as email text. Obviously not cool. However, Google needs to be explicitly clear on expectations and throw some people-power behind the reviews, since many are being denied by (seemingly) some automated process.
2. The second issue I see is that it will encourage less secure methods of using these apps. SMSBackup+ in particular is discussing the possibility of moving to "App Passwords" to bypass 2FA and provide the app access it needs to upload and store the data. Issue being, App Passwords are incredibly fragile, they provide near-unfettered access to IMAP and other account features with no auditing. Caveat emptor and all that.
I think SMSBackup+, specifically, has a bit of a gray line as SMS messages can technically be sent via email and vice versa, (among other similarities). It's a shame that Google is becoming so draconian about their data storage uses.
To me, it reads like this: Google is going to prevent users from storing users' data elsewhere. As if it's Google's data, not users'. Though with the free tier this as well may in fact be the case :-/
I also imagine there is the other side where the third party has crap data security. End users figure that their data is safe with Google and may not consider that there is a third party with their various levels of security.
I can see Google copping a lot of heat should Company X have a data leak and that data was originally gathered from a users Google data store, whether it is justified or not.
It's going to be "Gmail data breached".
These scam apps trade off being inside the protected platforms, so users expand their trust assuming (incorrectly) that a third party app will treat their data well.
"This is how scammers are now abusing Google Calendar to pillage your data"
"Gmail app developers have been reading your emails"
The headlines are ALREADY happening.
Why should google risk their brand so some grow fast and break things startup can create the next cambridge analytica scandal? They are one big CA type scandal away from being looked at as the next facebook (not a good look).
(Or, in other words, somehow all sense of fair play and decorum go out the window once we're anonymous on the internet. And this is why we can't have nice things.)
In this example "unlimited", which actually means "unlimited within reason" works perfectly well (even though it isn't well-defined) in a human setting. We naturally and instinctively understand that people don't mean "take as many as you want" or "make yourself at home" literally.
But on the internet, if it's a data/storage plan, we might get angry at anything less than infinity, because logically > "There is no point offering Unlimited storage and then stop people from using it."
I see this also apply to our "moral ease-of-use" for adblockers/paywall bypassers/torrents etc.
Google doesn't like storing files as e-mails with base64-encoded binaries because it competes with Google Drive. That solution - the Gmail Drive - existed long before Google Drive, and even had a nice tool that mounted your gmail storage as a network drive on Windows! GMail storage isn't unlimited, so I always considered it fair - they give me a couple of GB of free storage, it's up to me how I use it.
As for the unlimited offers and restaurants, people don't do that too much in meatspace because they'd get thrown out by security for obvious abuse. But they do it a little, like e.g. couples buying one cup and using it together. There are also natural limits to how much soda you can consume or use, even if you got away with taking home a whole barrel (sodas lose gas fast)...
(And note that the "decorum" and "fair play" doesn't apply in meatspace either, when it comes to e.g. retail chains making mistakes in their promotions, like that one famous case where (AFAIR) Lidl in Poland offered refunds for products you didn't like if you brought back the box, whether or not the product was still inside. You can imagine what happened next.)
However, ultimately, it's the company that's playing tricks on people with "unlimited" marketing, and they deserve the problems they get when people take it at face value (offering something with no intent to fulfill that offer is plainly dishonest). Reminds me of a mobile vendor that offered USB modem with free unlimited LTE for $notmuch, back when LTE was a somewhat new thing (~2012). A friend bought the subscription to test it out, and discovered that the "unlimited" LTE was actually throttled past 20th or 30th GB. Guess which company I never considered buying Internet services from since?
It's not because of customers that we can't have nice things. It's because of companies using dishonest marketing tactics and then acting surprised when some people call them on their bluff. It isn't so hard to say "no hard limits <small>but we throttle you past XX $unit, and there are following restrictions on use...</small>", except treating customers with respect is anathema to modern business.
Marketing does not create reality, no matter how much marketers may think otherwise. Words have meanings, you can't unilaterally attach some new one to a word and expect people to agree with it.
"GMail offers unlimited email storage, I can encode arbitrary data in an email. Therefore GMail offers unlimited storage! Wait, they banned me? HOW DARE THEY, FALSE ADVERTISING!"
Where back in the human world its not ambiguous at all what what Google, and every other service ever, means by this and is completely correct to call it unlimited.
The majority of the population will use these "unlimited" plans/products in a way that they never realize the limit. However there is always the outlier person that sees "unlimited" and is basically using the product at the max 24/7
Its much easier to say to the avg joe you have unlimited X instead of. Choose from the following 27 plans depending on how much a,b,c,x,y,z you need or even a you only pay per x of what you use! The avg person isn't going to even know those factors.
I think "Unlimited(asterisk)" marketing is here to stay for those reasons and if you are the minority power user then its up to you to read the asterisk
On a related note, I'd love to work for a company that offers unlimited vacation that's located in a country with decent protection against unfair dismissal. I wonder if I could get compensated for the (infinity-365) days of vacation a year I can't take?
The first is a lot of mobile US carriers. They have unlimited plans, but after n amount of data, your throughput is throttled. You don't even have to do something crazy like use your data plan as an ISP for you and your neighbors in your apartment. It's as plain as day when you sign up.
The other is Olive Garden's unlimited pasta offering. Some friends and I took this up as a way to kill time before a movie. We needed food, but we had two hours. Why not stuff our face til coma? Turns out that the first plate is a full portion. Every other portion thereafter is about ⅓ - ½ the size of the original (estimating), and judging by how long it took to get the 2nd and 3rd orders of pasta out, there's a soft time limit before they'll bring out your additional orders of pasta.
I understand why people want to be so skeptical about unlimited offerings, but are you really doing yourself any favors by intentionally spitting in the face of an offered service?
Sometimes. But sometimes, they actually disagree with the official/majority/whatever opinion, and this is this case. I disagree that the way "unlimited" is used in marketing is honest, or desirable, or should be allowed.
> Everyone knows what unlimited means in the dictionary definition and in the marketing definition.
Not everyone. That's literally the point of using this kind of language - some people will not know that marketers have their own dictionary that's different from the one normally used, and the way most of those people will use the service will not reveal the difference, so it's one of the cheapest lies the marketers can tell to pull in extra customers. It's a lie nonetheless.
> The first is a lot of mobile US carriers. They have unlimited plans, but after n amount of data, your throughput is throttled. You don't even have to do something crazy like use your data plan as an ISP for you and your neighbors in your apartment. It's as plain as day when you sign up.
It is, or it isn't. Where I came from, there are plans that offer you e.g. X GB of Internet, and then you're throttled. It's plain as day, says right so on the offer. Then there are other plans, that say "Unlimited", where what they really mean is ~5X GB of Internet and then you're throttled. It's dishonest, especially because those offers are created to make them look more competitive against real ISPs who do offer actual, unlimited Internet, usually by cable.
> I understand why people want to be so skeptical about unlimited offerings, but are you really doing yourself any favors by intentionally spitting in the face of an offered service?
It's called "voting with your wallet". Doesn't really work at scale, but still, it sends some market signal.
Life shouldn’t be about trying to take advantage of people or things to the maximum possible amount.
Edit: If I were to try to formalize the rules, I would say that the donuts are free for everybody in the office but not for anybody in the office.
If you are acting as a group with everybody in the office, which means behaving according to certain social rules involving fairness and sharing, then you count as an everybody and can have a donut. Once you cease to do so you no longer count as an everybody and cannot have a donut.
If you have special rights to the donuts, taking them won't get you judged. For example, the person who brings in the donuts can take the remainder home at the end of the day or may choose to give the rest to someone to take home, and there won't be any judgment. Further exceptions can exist on an office by office basis.
Tying this back to Google, I think there is one notable difference. Google is a private company, not a person, and is engaging in an extremely formal relationship by way of EULA/ToS/Privacy Policy/etc. Companies abusing loopholes in contracts are far more tolerated by people abusing loopholes in our shared social contract.
That is likely why my reaction at someone exploiting unlimited Google docs storage is far more 'meh' than someone violating social norms in the office.
When I was researching using a tool which leveraged a similar system and talked to a university which had backed up literally a petabyte of data to a single drive account.
Google's vague terms of service in terms of their "unlimited" storage is just a mess on both sides.
Like all cloud storage at the end of the day, if you're a paying customer or not, there are no guarantees you'll ever be able to retrieve anything once its off your infrastructure.
Yeah, until the Cambridge Analytica scandal revealed that agreements like this aren't sufficient to protect user data. I think Google's making the only acceptable tradeoff here.
(Also I guess they're already doing that, because the API isn't being killed off, it's being restricted to companies that threw a couple dozen kilodollars into a Google-approved bonfire.)
I still think there's a data sovereignty argument against the move we're discussing on this thread. That it's just friction makes the move less bad, but I think it's still bad.
The whole point of computing is being able to set up seamless interfaces for doing things you want done. It might be more secure that way, sure, but past some point security is opposite to utility. For me, it's just another reason I'm happy with moving my primary mail to FastMail on my own domain.
2) Even with a mechanism like the above, part of the issue with communication/social data like Gmail is that you can’t control your own privacy perimeter, because the other party could inadvertently leak your data through another app they connect
"Gmail app developers have been reading your emails" (The Verge)
"Google admits it lets hundreds of other companies access your Gmail inbox" (Telegraph)
"Google's 'Dirty Secret' Allows Third Party Apps To Read Gmail Messages" (TechTimes)
And all those articles were accompanied by appropriate ravenous Google bashing on this very site.
What exactly do you expect a company to do if the developers of HN and the "educated" tech media attacks it for having an open API?
In snail mail, a sent letter becomes the property of the recipient, and the recipient can do whatever he wants with it. Why should email be different? We're not talking about a static "profile" like Facebook has, but individual messages.
My working theory is that they were simply trying to refresh their fingerprint on my account since the only point of these alerts seems to be to get me to login using their web page. In particular I use Google for IMAP email, but never login to my Google account until forced. That's not so great for their metadata and tracking. Interestingly enough then when I then do login using a proxy half a world away from where I use my IMAP, Google never considers it a "login from a suspicious device." And yeah, as annoying as it is to migrate my primary email - Google is becoming intolerable on so many levels.
My guess is if some fingerprinting/Auth service is down, it fails in a 'safe' state, causing the login to be rejected and your account locked.
I agree with your sentiment, but part of the key difference for me is that SMSBackup+ is open source. I've been building the app myself and using it for years, so I'm very certain what it's doing and not doing.
This may or may not apply to the other apps being affected by this ban.
I'm very much hoping that we could resolve the issue with SMSBackup+ for the time being, but that's mostly up to the project owner.
I did hear of some folks using their own IMAP server and CALDAV service to target instead of Google, but I have not tested it.
The upside of the design is that it requires only one, very clear, permission: send emails to a given address.
Manually forward 10,000 messages to restore@smsbackup.com?
> I did hear of some folks using their own IMAP server ...
I ended up syncing to a Dovecot IMAP server on my local Linux desktop, which seemed to work just fine. I was apprehensive about the complexity of setting an IMAP server up, but that turned out to be misguided - Dovecot was incredibly simple to get up and running. I assume SMSBackup+ would work just as well as SMS Gate did here.
I eventually ended up moving from the stock messaging app to QKSMS (GPLv3, Github, Google Play, F-Droid) because it has built in backup functionality.
I suppose that people on the consumer side probably don't have that option, and it's probably not enough to get people to start paying.
All you need to do is guess the protobuf schema. Since protobuf is backwards compatible, you can be fairly sure your scraping won't break arbitrary either.
In many cases, they send back more data than is shown in the webUI too.
A) Google would die without that service
B) You're just fucking around and what your building could burn to the ground without consequence
https://killedbygoogle.com/ has 143 services listed.
C) You're looking to get acquired by Google
C) a) You're looking to get noticed and hired by GoogleI presume you can still download all your photos via Google Takeout right?
Google Photos should have been a frontend for Drive from the beginning. Splitting it into another storage is wrong.
Unless I'm missing something, it looks like the only way to do a full quality bulk export now is using Takeout.
If they're not only satisfied at data-mining my email, then screw it.
They say they'll fix it, but seeing the requirements I'm not sure they'll get it done in time.
https://github.com/jay0lee/got-your-back
Or the long term sustainability of such projects?
I've found gmail's own data export tools to not work at all for any inbox of a considerable size (100gb+) - so third party tools are the only way to actually back up / migrate email data.
Without such tooling, relying on Gmail would be a huge mistake for anything remotely important.
GYB has such functionality built-in since 1.20, see the release notes: https://github.com/jay0lee/got-your-back/releases/tag/v1.20
For gmvault, you have to create it manually: https://github.com/gaubert/gmvault/issues/335#issuecomment-4...
Google could have added a contract that would plainly state that any data needs to be wiped out etc and enforce that contract if anything is fishy.
Google could have created a process to clearly inform the dev that the user wants to delete google related data and impose deadlines on it.
Those are simple, but I think Google was just lazy and listened to a bunch of lawyers instead of thinking out the box.
I have an app that allows to link your email account thru Nylas (with google), now I would have to pay the security audit? No way. I told my customers that any google account that is not a GSuite which whitelisted the app (most of my customers corporate) that they might have warning dialog when connecting their gmail account. There is a limit of 100 linked account without verification ;(
The, I'm sorry, WTF? This is not Google's data.
I'm afraid I have no idea about GSuite accounts, you'd have to ask somebody who knows Google's policies.
The lock down is for the Gmail API especially for API that allows reading user’s email.
Any App has to get OAuth 2 token to get access to the API. The user has to explicitly provide access . The approval screen will show each type of access the app is asking. See an example here [2]
In addition, Google will send an email to the user immediately after the approval, with a scary warning.
The user can withdraw the app access anytime, from Google account page.
The data access concern Google is projecting is that the APP can read user’s email (Remember, the app can read only those who explicitly gave the app the permission to read their email). The “lockdown” is a direct reply to the media frenzy that “Gmail allows any app to read anyone's email” [5]. Gmail does not allow reading email automatically. The user has to allow explicitly.
In order to get Gmail API access, the app has to go through a Google review process where Google will ask the developer to justify each type of API access the app is requesting in addition to explaining (with videos) what the app does and how the API is used. The first level of approval process demands you to publish a comprehensive privacy policy and in my experience, anything like “marketing” or “research” in the privacy policy will get you disapproval. [3]
Such a strict approval process is good and fine, and well appreciated till this point. The issue comes for the last part of the approval process.
Those Apps that requires read access to Gmail has to get themselves assessed, through Google appointed third party security assessors paying $75000 USD annually.
This is the main blocker.
This will kick out any app or add-on that small scale developers create. It will block new entrants. What remains will be established apps that are generating huge revenue to justify the “protection money”. They get an added advantage that there will no longer be any new competition.
It is not the restrictions, or the intention to protect the end user that is in question but the “first save my back” attitude in the process, and the bait and switch - that is the problem. In summary it happened like this:
Hey developers come, build apps using our platform, show your innovation! Developers start investing time and effort on the platform, approval process is smooth and fare Somewhere else, someone misuses someone’s system, huge media attention Sorry developers, you go to Mr X , keep paying him and we will keep you here. If not, trash your product and go away.
[1] https://medium.com/@prasanthmj/lessons-learned-developing-an...
[2] https://www.youtube.com/watch?v=GGXFQUmZTf4
[3] https://blog.gsmart.in/applying-for-g-suite-api-approvals/
[4] https://medium.com/@prasanthmj/google-restricted-api-scopes-...
[5] https://www.wsj.com/articles/techs-dirty-secret-the-app-deve...
I get why they are doing it but blah, now I have to find solutions for everything again.
Maybe email clients will go back being email clients with IMAP so they can be used with _any_ provider, not just gmail.
[0] https://cloud.google.com/blog/products/identity-security/enh...
Even explorer was less tightly integrated 25 years ago...