Ask HN: Why is YouTube adding “&pp=sAQA” to video URLs? | Hacker News Reader

Ask HN: Why is YouTube adding “&pp=sAQA” to video URLs?

109 pointspdkl9569 comments

YouTube started adding a new parameter "pp=sAQA" to video URLs on most index style pages (e.g. /feed/subscriptions, search results, the /videos page on any channel). The actual video pages (/watch) strip the pp= parameter, and it doesn't appear to be added to the URLs for the "recommended" videos.

Does anybody know what this parameter does and/or why it was added? It's really annoying; using YouTube URLs in the shell now requires quoting due to the "&".

View on HN

It's fairly annoying for other reasons than just shell quoting. For instance, I use the browser history to know whether I've already watched some video; this extra parameter makes all these video links appear to be unvisited to the browser (since back when I visited them, that useless extra parameter didn't exist).

It's not as good, but the history is pretty useful: https://www.youtube.com/feed/history

The absolute future of a proprietary service-specific history.

That link shows empty to me, since I've never logged into YouTube, and most of the time I'm not even logged into the Google account.

(I don't know if it's still the case, but I've heard that if you log into YouTube, and Google for some reason decides your real name is not your real name, you could lose access to your whole Google account; and at least to me, the Google Talk account was more important than being able to comment on YouTube videos.)

I don't think that's been a thing for a long time.

https://en.wikipedia.org/wiki/Nymwars

Last I checked, it was impossible to create a YT/Google account without providing a (working) phone number. Which is even worse than requiring "real names" (which can at least be spoofed).

That depends. I’ve found that, while creating an account on an iOS device, providing a phone number isn’t required, but while creating an account via a desktop browser, it is.

It will probably depend on other factors as well such as IP, country, …

It was still possible as of six months or so ago, who knows what factors Google takes into account to decide whether to require a phone number or a simple email.

That would require to be logged in to a Google account

Does it work when a video was deleted?

how do you use the history exactly?

I inject this into YT

    // make VISITED links distinct
    //subscriptions/lists
    css += 'a#video-title {color: #167ac6!important;}';
    css += 'a:visited#video-title {color: #141761!important;}';
    //commenter channel name
    css += 'a#author-text {color: #167ac6!important;}';
    css += 'a:visited#author-text {color: #141761!important;}';
    //sidebar
    css += 'a #video-title {color: #167ac6!important;}';
    css += 'a:visited #video-title {color: #141761!important;}';
    
    css += '#content-text a, #description a {color: #167ac6!important;}';
    css += '#content-text a:visited, #description a:visited{color: #141761!important;}';

makes pretty clear which links are visited

Yesterday after discovering &pp=sAQA breaking by History tracking (got a custom CSS injected to make sure its distinct) I added this

    document.querySelectorAll('a[href*=\\&pp\\=]').forEach(el => {el.href = decodeURIComponent("/watch?v="+el.href.match(/v=(.*?)(&|$)/)[1])})

to my userscript injected into YT. I still need to pinpoint reliable DOMSubtreeModified event to make it work without manually reloading the page.

Edit: and just now discovered its being added server side. Fetched https://www.youtube.com/youtubei/v1/browse?key=xxx already has &pp=sAQA, makes patching even easier, just donna hijack fetch and strip them in the handler... aaand done:

    const origFetch = unsafeWindow.fetch
    unsafeWindow.fetch = async (url, init) => {
      if (url.url.startsWith('https://www.youtube.com/youtubei/v1/browse?key=')) {
        let response = await (await origFetch(url)).text()
        response = response.replace(/(\&pp\=.*?)",/g,"\",")
        return new Response(response)
      }
      else {
        return origFetch(url)
      }
    }

    window.addEventListener('DOMContentLoaded', ()=> {
      if (document.location.pathname == "/feed/subscriptions") {
        document.querySelectorAll('a[href*=\\&pp\\=]').forEach(el => {el.href = decodeURIComponent("/watch?v="+el.href.match(/v=(.*?)(&|$)/)[1])})
      }
    })

Regular expression replaces on content are very fragile. Unfortunately, browsers do not yet provide general request regulation capabilities. Until then, Transcend Consent Manager's API can solve this much more easily with a more robust request-override based approach:

    self.airgap = {
      overrides: [{
        matcher: '//www.youtube.com/',
        override(request) {
          request.urls.forEach((input, i) => {
            const url = new URL(input, location.href);
            if (url.searchParams.has('pp')) {
              url.searchParams.delete('pp');
              request.urls[i] = url.href;
            }
          });
        }
      }]
    };

Its true URLSearchParams is a proper blessed way of working with url parameters, but its just my tiny userscript working on a serialized JSON, so either I grep and be done with it, or be forced to actually parse it, traverse huge Object looking for 100 individual URL fields to modify "properly", and stringlify once again. Im good with regex thank you :)

>Transcend Consent Manager's API

what is that?

>request-override based approach

the idea is to modify those URLs before they get rendered on the page, this way they get applied :visited style in the first place.

> what is that?

A new consent management tool for website owners. We're currently in closed beta. Learn more here: https://transcend.io/blog/defeating-cookie-banners

> the idea is to modify those URLs before they get rendered on the page

These overrides are applied before the URLs are rendered on the page or trigger any requests.

>DOM nodes by directly patching the prototypes, methods, and accessors of all global element constructors and DOM tree construction utilities. We can write a patcher which overrides the constructor prototype, which allows us to check if the URL is allowed before the DOM mutation is performed.

Now I understand. Your product is aimed at corporations seeking more mature? compliance with privacy laws. Sounds like tough market.

I actually find all kinds of browser history pollution a very serious problem rendering it almost useless (only useful the way "recycle bin" is - to recover an address of that useful page you have seen yesterday but forgotten).

I wish there were browser extensions capable to smartly sanitize the history by removing duplicate URLs pointing to the same actual page visited multiple times during the day and/or by via slightly different URLs, removing the specific parameters which can be removed without changing what does the link point to, rearranging the remaining parameters alphabetically, removing GMail and alike useless URLs, also normalizing the entry titles by removing everything but the actual article title (e.g. removing website name or moving it to an uniform place - always the beginning or always the end) etc.

I like the way Vivaldi Browser presents history. So much easier to use than other browsers.

I haven't used Vivaldi before. Can you explain what is eaiser to use?

It's just organized better. There are some screenshots on the help page that show it better.

https://help.vivaldi.com/desktop/navigation/history/

I'd prefer that the browser be taken out of the equation completely.

The web browser is becoming just a shitty operating system, largely controlled by unaccountable megacorps who are primarily interested not in empowering users but in tracking them and serving them ads.

You can use curl if you don't like browsers. It just needs a lot of imagination to get an idea on how the site is supposed to look like.

Whatever, pmoriarty is right. Today a browser indeed is an operating system with its own GUI toolkit and a decentralized repository of apps (we call the web, most of the sites actually are apps today).

Perhaps if the web servers served something like QML/XAML/whatever (while other web servers serving just APIs for them to get/post the content) and there was an operating system command to run such an app by its URL, both the developers and the users might have a better experience. Or perhaps worse.

This way we might even have more than one standard competing: QML, XAML and whatever else would emerge (introducing new would be more viable) vs just HTML.

So YouTube is setting environment variables for people who don't do shell quoting. Next they can add "&reboot" to all their URLs.

This is how you append query params to a url - nothing to do with the shell.

The GP complained about shell quoting.

I doubt it is YouTube's intention to require people start quote URLs in shell, though.

....&foo=BAR

sets a shell variable, not an environment variable. The value of $foo will not be available to shell-forked processes.

What? Look at the URL in your browser right now. It has a bunch of x=y parameters. That's how URLs work.

The parent comment is talking about what would happen if, for example, you typed this in bash to try to download a video:

$ youtube-dl https://www.youtube.com/watch?v=dQw4w9WgXcQ&pp=sAQA

In this case, bash would interpret it as "run youtube-dl in the background with the parameter 'https://www.youtube.com/watch?v=dQw4w9WgXcQ', then execute the command 'pp=sAQA' to set the 'pp' variable".

The comment is joking that if YouTube decided to replace "pp=sAWA" with "reboot", they could get a lot of people to accidentally reboot their machines instead because they didn't quote the URL.

That was always a problem if you just copy-and-pasted URLs into your terminal. You need to quote them. It's not new.

It used to work in bash but doesn't in zsh

curl http://foo.com?x=1&rm -rf ~/

That's hilarious

From the source the pp is - "playerParams":"sAQA"

Its being added server side. YT does a fetch to https://www.youtube.com/youtubei/v1/browse?key=xxxxx and the response is subscriptions list, all already with &pp=sAQA attached

Maybe it's A/B testing? if so, I'm in the same "sAQA" bucket ...

An analyst somewhere who isn't aware that people are literally playing with the test buckets and jumping between them to figure out what the query parameters do is going to write a confident report on why A outperformed B.

Weird they need URL params when they know who you are anyway from the browser thumbprint :-)

Sometimes the intention is to detect the param on the way back, regardless of whom it came back from.

For example, the param may point to specific time on the video.

Or the param may point that the video is part of some playlist, so that when somebody clicks he is brought not just to the same video but also to same playlist.

As we don't know anything about this particular param we don't know if it is the case.

They're 80% assumptions and fiction in any case, so it's OK.

What fraction of the YouTube userbase do you imagine manually edits URLs?

I always strip youtube links of an ampersand and everything that follows when I give them to anybody.

especially fbclid

If I forget to do that I like to say "pardon my url"

I manually edit "utm_" URL parameters to at least feel like I'm sending harmless easter eggs to analysts somewhere. utm_campaign=helpimabot

Congrats, you’re in the 0.0001% of youtube users who actually does that

That number is way too high.

Most likely quite low. I let my addons handle that.

https://gitlab.com/KevinRoebert/ClearUrls

Didn't work...

Some.

Stripping is nice an all but I want a plugin or greasemonkey script to rewrite that to pp=random on every request.

I'm seeing them on the home page, channels, explore, subscriptions but not on searches and 'related videos' while viewing one.

/edited for fullness

I'm seeing it on all pages... Home, Explore, Subscriptions, even History.

I see it on the front page, but not on left sidebar when watching a video or in the overlay that shows other videos once the one you watched is finished.

I see it in Explore and in all Subscriptions categories(not Subscriptions feed/subscriptions).

It is also not appended to mixes/playlists for me.

Does show up in Library and History, not in Watch Later and Liked Videos.

Shows up in: Movies & Shows, Gaming, Live, Fashion & Beauty, Learning and Sports.

Sounds to me like a side effect of an A/B test.

That gives a plausible cause to my suspicion that I'm seeing a bunch of stuff I already watched showing up in suggestions again.

Why would this affect YouTube’s internal recommendation system?

YouTube has never been shy of recommending already-watched videos, the issue is whether you can see that you watched them. I think.

Seems like it's being appended to video links shown on youtube that are not from either search results or related videos.

I suspect this is a way for them to exclude click metrics, etc. from videos that were not shown as a result of machine learning/lexical analysis/whatever.

I'm sure they've been identifying how videos propagate since forever, but my guess is that they recently decided it would be efficient to add some cryptic-looking parameter (that probably just corresponds to a boolean) to all video links shown on channels, the home page, etc.

atob("sAQA") = "°\u0004\u0000"

Although not happening for me

Shooting from the hip here, but this ("\xb0\x04\x00") could be interpreted as a protobuf with this IDL:

    message Foo {
      optional int32 bar = 70;
    }

that decodes to the following value:

    bar: 0

(or bar could be bool of value false, or enum at value 0/UNSPECIFIED, or int of any other size)

But with such a short message and without other samples it's difficult to prove or disprove that this is indeed a protobuf :).

One could try to set &pp=sAQB (bar = true/1), &pp=sAQP (bar = 15), ... and observe what happens.

This seems the most like case to me. Google loves their query PBs.

Why don’t we have better magic in shells for quoting pasted urls?

Not sure if its something related, I got a wikipedia link from someone which included `&wprov=sfti1` as query string as well.

Unrelated. The Wikimedia Foundation has actually documented the wprov query parameter. It's used to track the source of a link; "sfti1" is "Share a Fact on iOS for text shares, feature version 1".

https://wikitech.wikimedia.org/wiki/Provenance

TIL

Reply on news.ycombinator.com