Hacker News Reader

Comment by dleslie | Hacker News Reader

This has nothing to do with security and everything to do with banning tools like youtube-dl, wget and others; from the post:

> The browser must identify itself clearly in the User-Agent. The browser must not try to impersonate another browser like Chrome or Firefox.

> The browser must not provide automation features. This includes scripts that automate keystrokes or clicks, especially to perform automatic sign-ins.

Edit:

I feel like mentioning youtube-dl was a mistake. It's not just about youtube-dl.

This policy bars the use of all text-based browsers, headless browsers, browsers without javascript, and browsers with automation when accessing Google's services. It bans browsers that contain Node.js, even.

View on HN

> The browser must identify itself clearly in the User-Agent.

I wonder what they'd think of my proxy which I have setup to (among other privacy respecting features) rewrite the User-Agent to "By allowing me access, you waive all rights and policies regarding my access." This is basically my form of EULA.

> The browser must not provide automation features.

LOL. This was obviously written by some tech illiterate law type, perhaps a first year law student? I fear to think what incompetent engineer working at google of all places would have come up with that verbiage . . .

Yeah, the browser itself is an automation feature. It automates the downloading of a file over HTTP, downloading of any dependent resources, managing of caches, and rendering of the result. That these are so frequently done that this automated pipeline is referred to as "a browser" doesn't mean that this isn't an extremely automated system.

> To protect our users from these types of attacks Google Account sign-ins from all embedded frameworks will be blocked starting on January 4, 2021.

(emphasis mine)

So I don't follow how this would have anything to do with banning youtube-dl, which doesn't require login? And as the blog post mentions, you can still bootstrap auth through a normal web browser, and pass the auth token to your command line / less secure browser / ... app.

(Disclaimer: I work at Google, not on anything related to this blog post or to your hypothetical scenario.)

Passing oauth tokens into automation tools is a common use case in order to automate the retrieval of account-restricted content.

What does that have to do with youtube-dl? (Sorry it's been like 5 years since I used it, I don't remember that being required)

It allows one to download private videos that your account can access.

Such as:

- I have a script that downloads my liked videos (in case they get deleted, which I’ve found out happens a fair bit)

- I also have a script to download my watch later videos (for sync to devices without YouTube Premium/Red/whatever)

Would you share those scripts?

Sure:

```bash

# !/bin/sh

cd /media/youtube-dl

docker-compose run --rm youtube-dl -v --cookies /etc/youtube-dl.cookies.txt https://www.youtube.com/playlist?list=INSERTYOUROWNWATCHLATE... -o "watchlater/%(title)s-%(id)s.%(ext)s" --ignore-errors

```

That’s a bash script that runs via cron. One thing to note: this uses the cookies from a logged-in browser session because at some point YouTube blocked password log in from youtube-dl. This was is a bit of a pain to set up, and I wish it was not the case, but it mostly works.

So the theory is this has nothing to do with security, but is only used to break private video downloading of youtube-dl?

It blocks misrepresentation of agent, in general; automation is also blocked in general, but _especially_ for authentication.

Sure but if the goal was to block youtube-dl usage, wouldn't they target the vastly more common usecase without authentication?

There's an ocean between required and useful.

Which would still be fine. The only thing that'd be blocked is obtaining those OAuth tokens by passing your Google username/password to a browser automation tool.

Which would break some common authentication options in ytdl:

https://github.com/ytdl-org/youtube-dl#authentication-option...

Let's dont act as if the whole world was trying to fight against ytdl.

It's not just about ytdl; this policy will ban all headless browsers, text-based browsers, and browsers with automation tools.

And it's also ineffective, because any full GUI browser can be automated with extensions or userscripts, so banning useragents or non-javascript browsers won't actually prevent automates sign-ins.

It is not the whole world attacking ytdl - but Google, definitely.

Evil by default?

when you delete "don't be evil" tagline, you become evil by default

So they're breaking mountains of automated test and scripting tools? Nice to know.

>Headless browsers locked out (thanks for nreaking test automation) >Node.js (riiight) >Text-based browsers (Going aft Lynx? Of all things?) >JavaScript MUST be enabled

Totally not a power grab here folks. Totally not a company known to value reaping user data in any form possible up to any kind of user hostile behavior, or exercising undue influence over the character of the Net.

Nope, no siree, Bob. Moving riiiight along.

How does youtube-dl obtain the token today?

https://github.com/ytdl-org/youtube-dl#authentication-option...

Username, Password, 2FA, etc.

And you claim that doing more to stop people from giving their google account password to "random apps" (I personally trust youtube-dl a lot too, but "random apps" is what it comes down to) and forcing those apps to use OAuth to obtain scoped tokens has "nothing to do with security"?

If that were all that they were doing I might agree; but they are blocking browser identity misrepresentation and automation, as well; it also requires that all "browsers" have a complete implementation of web standards.

It explicitly blocks "headless" browsers.

> You must confirm that your browser does not contain any of the following:

> Headless browsers

> Node.js

> Text-based browsers

Security for whom? Locking the user out of the software they want to use is not improving security for them.

That's only true if you assume the user is perfectly capable of evaluating the trustworthiness and quality of the software they want to use. It's understandable that that's not the assumption Google designs their security under. Yes, that sometimes somewhat sucks for us power users.

That’s a pretty fake excuse IMO as long as the browser keeps rendering web pages that look like Google’s sign-in page.

Definetly not on the right side of the history. With all the social changes and uprises we see one would expect you to do better.

It has everything to do with security: securing Google's control.

Google wants to take over the Internet. We should not let it use these "less secure" excuses to sway the public opinion.

Google’s control...over the security of Google accounts?

If you are worried enough about Google’s dominance over the Internet to be upset by this particular practice, it is unlikely you have (or should maintain) a Google account.

I’m not a “Google stan” by any means, but to say that they want to take over the Internet is just not true.

> to say that they want to take over the Internet is just not true.

I don't know if they "want", but they already do have control over various aspects of the Internet, especially of the web, but also DNS and email.

Google wants to dictate what user-agent you can use to access their sites: that's pretty controlling!

Can you imagine if Fox announced that only approved televisions could show their content?

"The browser must not provide automation features."

It would be interesting to see this examined in the context of accessibility requirements created by the ADA.

This is kind-of funny requirement given the history of user-agent strings being incredibly convoluted.

I mean Chrome doesn't clearly identify itself as Chrome, it still identifies itself as Apple Webkit

Also given Google's history of sending corrupted version of pages to Firefox useragents.

https://www.reddit.com/r/firefox/comments/7whdv4/googlecom_i... https://androidforums.com/threads/problems-with-firefox-user... https://www.ghacks.net/2018/07/25/google-making-youtube-slow... ... and many more

Well I guess my unofficial YouTube chat bot won't work anymore. The YouTube API is awful compared to the Twitch one for bot creation so it is easier to get the functionality you want using Selenium.

Why not? This restriction was added on Android years ago and it basically means that you retrieve the OAuth2 token with a normal browser and then send it to your automaton script/app.

It blocks auth to prevent phishing, not actual access.

If you'd bothered to read a little more before knee-jerking a reaction comment, you'd know this is only for the authentication flow.

I did read that; did you know that passing oauth tokens into such automation tools is commonplace?

OAuth tokens used in automation tools will continue to work. Entering in username & password through auth, to automate an OAuth flow (or any other traditionally manual flow) will stop working. Breaks some puppeteer scripts too - but those have been getting flaky for a while now.

Thus making it even more cumbersome for users; now they simply login, in the future they'll have to know how to get the oauth token.

And, OAuth tokens can be revoked meaning scripts will just suddenly fail.

What's your point? Passwords can change and sessions can get invalidated, which all has the same effect.

Yes I would agree with that, except that if you change a password you know the scripts will fail, but if an OAuth token gets invalidated by the system and not you, then it will fail without warning.

And if your password gets reset by the system and not you, same story.

What makes you say oauth tokens are any less robust? Aside from the fact they usually have an expiration attached to them, there's not much difference.

Also fair point.

What makes me say that? Experience.

However, I will say to counter my own point: it’s not all roses. Using password by necessity means that your password is now stored somewhere that is likely more easily compromised (In my case: secure passwords are stored in 1Password, but passwords for script usage are either stored in an ENV or in the script itself, neither of which are great from a security standpoint)

It's OAuth. The application can launch a normal browser for the OAuth flow and have the user complete it.

For plenty of applications the whole purpose is not to run "a normal browser" and possibly not even have it installed.

You can also use a browser on a different device if your thing can't run a browser itself. OAuth covers a large space of options.

They can spit out a url for you to copy into a normal browser, then.

And? What if I want to automate my login flow?

You'll need to find another provider which doesn't care that much about preventing phishing attacks. Google accounts are a big target so it makes sense you move away from the masses.

In practice it just means faking the user-agent and other fingerprinting more enthusiastically. I'm not sure how google can win that without resorting to the same anti-cheat measures as games companies.

You'll try, but the first time you won't know what fingerprinting tests they are going to do. After a few iterations you'll succeed, but it will be obvious to Google that the account you've just been testing it on belongs to someone trying to break their auth restrictions...

Good luck keeping your account!

> The browser must not provide automation features. This includes scripts that automate keystrokes or clicks, especially to perform automatic sign-ins.

If a web developer knows what they are doing they are using the standard web APIs supplied by the browser in an efficient way, designed to be invisible to accessibility for accessibility test automation, and thus this control from Google is largely unenforceable. As such I believe this is just a block against incompetent forms of automation that probably shouldn't be there in the first place.

Pretty much. It also helps their ad business to combat fraud.

This is probably the only case that actually affects them. There's really no argument for security here.

The announcement specifically says this:

> ...Google Account sign-ins from all embedded frameworks will be blocked starting on January 4, 2021

It says nothing about non-login related actions.

The whole premise is a mistake, not the mention of youtube-dl. This policy doesn't ban the the things you claim it does. It is not 'everything to do with banning wget'. It's just a strange and mistaken conclusion you arrived at, seemingly by very selective reading.

> his has nothing to do with security and everything to do with banning tools like youtube-dl, wget and others

Exactly. What's insecure about an application that can establish a secure connection using an accepted version of TLS and cipher?

Google has went rouge for quite a while. AMP was another example of the same nature.

Has anyone thought of just not using Google? By using them you give them power in your life

It's to stop scraping of Google Data.

There is currently millions -> hundreds of millions being made by scraping Google content.

Kind of hypocritical given that Google's entire business is founded on scraping the rest of the internet.

Reply on news.ycombinator.com