Hacker News Reader

Comment by grishka | Hacker News Reader

Huh? Userscripts are dead according to author, but I apparently never got the notice because I have about a dozen of those things installed right now. Three of them are for YouTube because it's this much annoying as is.

View on HN

They aren’t dead but with more and more build systems that obfuscate everything in the page, including CSS classes and IDs, it’s sometimes hard to do something really useful.

Would you mind giving me an example of something you've wanted to do that is really hard (or even nearly impossible) due to the these issues (so I have a standing concrete challenge case)?

As an example, I wanted to create a custom, more compact style sheet for JIRA (I don’t understand all those products with complex interfaces AND white space everywhere). I failed. It’s a PITA to write any selector and it wouldn’t survive the next build from the devs.

You don't have to select things based on CSS selectors. If you something really resilient, you select something based on the content of the elements (a button that says "Send" will rarely change to say something else, while the styling/DOM can change a lot). Otherwise xpath is another option of the DOM doesn't change a lot while CSS class/ID attributes changes between builds.

I have another fun challenge, try to completely disable the auto-zoom-feature on Google Maps. Last time I tried, I stepped through 20 stacks of obfuscated nonsense before eventually giving up.

But tbh I can't really imagine any scenario that is near impossible - except for WASM driven web pages which will break just about anything :(

why would WASM driven pages be any different to deal with than non-WASM driven pages?

You can't scrape HTML out of a rendered bitmap. It's not the only mode WASM can be used, but often all the rendering happens in the compiled code and the page only receives a series of images to display on a canvas. It's Flash all over again.

> It's Flash all over again.

Flash was at least easy to decompile. The decompiled code was so good most of the time that you could compile it back into a working swf, with modifications if you want any.

But then both WASM and Flash have to get the data from somewhere. That is usually something resembling an API endpoint that returns the result in XML or JSON. Why not sidestep the whole client app thing and go straight for that API endpoint?

Almost nobody is doing this, and almost nobody ever will because it breaks accessibility and a whole host of other features. I would save the alarmism for another day.

You're right about the accessibility and other features breaking. You're wrong in saying that (almost) nobody would do this. What (almost) nobody will do is creating an HTML-based rendering engine for windowing (GTK, Qt, wx, etc.) and graphical (SDL, DirectX, OpenGL, etc.) toolkits. It's technically possible, but maintainers have enough work with supporting just the 3 desktop and 2 mobile platforms. In the absence of a proper HTML rendering frontend, there's really nothing you can do other than compile the whole graphical stack and render into a canvas. That's the default for graphical apps under Emscripten - all games and GUI apps currently work that way. The only apps that have a HTML rendering frontend are CLI tools and REPLs, where it's trivial enough to rewrite the rendering in JS. There are frameworks currently being developed that allow rendering of widget trees in HTML or SVG, but they're still immature and not widely used. Well, WASM itself is not widely used, so it's normal.

BTW: were you around in the days of Flash? Because it broke accessibility and a whole host of other features the same way, yet it was still used and hugely popular (there was also ActiveX, which worsened things even more). It began with online games and graphics-rich pages, but it evolved frameworks for generic apps (ie. Flex) and then even CMS engines. Soon enough Flash was used for even blogs and e-commerce. The situation today is different due to the Web having evolved into a more capable and standardized platform, but there's no shortage of devs who would welcome the ability to run their app in the browser, but who dislike HTML/JS so much they would do basically anything just to not have to touch it - accessibility and a lot of other things be damned.

Don't underestimate how irrational people and even whole communities can be.

As a bonus, here's an example of the "almost" part: http://35.158.218.205/experiments/webDOOM/

Anatine[0] was a Twitter client maintained by Sindre Sorhus, a popular open-source maintainer. It made use of a userscript to add extra functionality to Twitter. Sadly, this meant keeping up with Twitter's ever-changing DOM class names.

It was eventually abandoned.

[0]: https://github.com/sindresorhus/anatine

Facebook and Instagram ads. Impossible to block through CSS now as the CSS classes are dynamically generated + it's even hard to pick out sponsored posts using .js too.

I am pretty sure that there is a team of engineers whose job is to office obfuscate ads on Instagram and Facebook.

Surely these people can find something more beneficial to society to work on, such as selling heroin to middle schoolers.

Money is money. Few of the jobs I’ve worked have had a positive impact on society.

From js, simply find all <a> tags that have a url containing "utm_medium=paid". Once you get them, for each tag element go up parentElements until the parent is <div role="feed">. Then remove the child.

Just block the entire feed and go directly to the profile pages of users you want updates from.

ca. 2014 I was able to write a bookmarklet that scraped a twitter thread by accessing the DOM (even causing page navigation) and rendering it as a reply-tree.

Two things make this vastly more difficult now:

1. Twitter uses obfuscated css classes that appear to change on every deploy, which makes scraping hard.

2. You used to be able to get around the limit on bookmarklet code size by adding a script tag to the page and loading remote javascript. Now browsers respect HTTP headers that sites can use so that only scripts from whitelisted sites are loaded.

Do older tweets get these new css classes when viewed after a deploy?

I could see using a set of standard tweets you created to sort of benchmark the change.

I imagine they would. That would be a good approach for data scraping, but difficult to do in a bookmarklet.

Try it on a Tailwind & React based site. Lucky for a site like that it used NextJS so it has a lot of JSON state dumped in plaintext.

I am asking for a concrete task challenge, not a kind of site or technology.

Ok scrape data from manifold.markets

Does it change a lot? Because it's pretty easy as is... eg.

% curl -s https://manifold.markets | htmlq '#__next > div > main > div.items-center.flex.flex-col > div > div.gap-8.flex.flex-col > ul > div > div > div > div.relative.flex-1.gap-3.pr-1.flex.flex-col > div.peer.absolute.-left-6.-top-4.-bottom-4.right-0.z-10 > a'

will get you all the anchors for the stories on the home page, and from there it's easy to get any other part of the stories.

It changes quite often. Since tailwind is utility based and not semantic it will likely break within the week.

Probably Tailwindcss.com, then?

No, like "the website X is heavily obfuscated but I wish I had a userscript that could make Y modification". I work on runtime code modification tech, and am mostly known for my work on iOS and jailbreaking. Someone saying "there exist apps that use this weird compiler that are difficult to patch" isn't useful: I need a specific app and a specific target like "the Orchid app is using Flutter, which has its own weird Dart to native compilation step, which is making it almost impossible for me to use the tools we already have available to add a button that automatically builds multi-hop accounts".

Trying to block Google’s “people also search for” div (when you navigate back to the search results page) is not easy IIRC

You can try these uBlock filters. Difference is night and day

    www.google.com##div[role="heading"]:has-text(/^People also ask$/):upward(5)
    www.google.com##span:has-text(/^People also ask$/):upward(5)

    www.google.com##div[role="heading"]:has-text(/^Cast$/):upward(5)
    www.google.com##span:has-text(/^Cast$/):upward(5)

    www.google.com##h3:has-text(/^Videos/):upward(5)
    www.google.com##h3:has-text(/^Images/):upward(5)

    www.google.com##div[role="heading"]:has-text(/^People/):upward(5)
    www.google.com##span:has-text(/^People/):upward(5)
    www.google.com##h4:has-text(/^People/):upward(5)

    www.google.com##span:has-text(Related searches):upward(5)

    www.google.com##span:has-text(/^Latest/):upward(4)

    www.google.com##h3:has-text(/^Top stories/):upward(6)

    www.google.com##div:has-text(/^Trailers/):upward(4)

    www.google.com##g-scrolling-carousel

This thing drives me nuts! It’s always popping up JUST at the right time to catch my click when I’m aiming for the second search result. It’s been that way for long enough now that I’ve started to assume it’s like that on purpose, just to keep you from leaving.

Override the font on all YouTube text elements.

You'd probably think it'd be simple, right? This being the kind of thing that Cascading Style Sheets were designed to handle elegantly and gracefully?

https://maya.land/user-styles/youtube/

I always thought people on here were just cranks complaining about style bloat, because a lot of the "this could have been a text file" arguments were applied to things that were genuinely better with the presentation polish. But this? How can developer ergonomics be worth this?

Anyway, user styles and user scripts and bookmarklets are all great and I use them all the time. This level of Big Tech deformation of the basic web technologies makes them more annoying, but still not impossible.

There were these things called API spies back in the VB6 days that would figure out the appropriate way to refer to whatever you clicked. The one from PatorJK would even produce code to do things to it from downloadable templates.

It's still around: https://patorjk.com/blog/software/

I wonder why no one ever made something like this for the web.

It is built into every browser's dev tools, but the point GP was making is that the way many websites are built now means that for an external observer, the way you refer to an element changes with every reload, or with every new build (which may happen at any time and frequently for actively developed sites).

What do your user scripts do if you don't me asking?

On a site note: Been looking for one to show reccomendations always based on simmiliar vids, not my recently watched and whatever the black box algorithm is offering me. Unfortunately I never figured out how do it properly, since the position of the element seems to be random and sometimes straight up missing.

The YouTube ones?

1. Prevents channel page video from auto-playing (not mine)

2. Removes the stupid custom scrollbar that takes up actual pixels on my screen at all times

3. Automatically sets highest available video quality by clicking through the quality menu so I don't have to

And those are in addition to SponsorBlock and uBlock Origin extensions.

I made one for youtube that moves the seek bar below the video, instead of on top of the video. In addition it removes the gradient etc. I've always hated how it covered up the content. (pure css, but still)

The one that switches off the comments section makes the platform quite a bit more pleasant

I use one to redirect youtube shorts pages to the regular youtube UI. Also one for old.reddit.

Many sites are turning on CSP, a web security capability which lets sites block loading content from anywhere they don't allowlist.

Userscripts will run, but if they try to load any content from elsewhere, that breaks. So, for example, the Hypothes.is extension wont load. Affected sites keep growing. https://twitter.com turned on CSP many years ago.

This is such a silly problem to have. User agents that want an exception to a policy are free to make one.

The mainstream browsers & spec authors sneer at & detest userscripts, wish they'd never been a thing. https://github.com/w3c/webappsec-csp/issues/444

I agree. This is simple & stupid a problem. It's grossly anti-user. But today, security rules, and security sees all the misdeeds of users, and those who would prey on them, and they have no emapathy left for creativity and fun. There's no moral-value-party left anywhere that defends user freedom, that is pro end-user-hacking.

The Manifest V3 (MV3) catastrophe is still scheduled for 4 months out, even though there is still only 1/3rd the work done thay it would take to allow GreaseMonkeyTamperMonkey/et-cetera to live. The new browsers are actively unwinding power & possibility, forever trying to undo the permissiveness that it had (ex: CORB, and now ORB). The standards authors & browser makers are infected with deep deep fear. Certainty & closedness is winning.

Keep in mind that all of the mainstream browsers are funded by companies that make money off of advertising, so you should always look for ulterior motives from them.

+1, I use somebody's Greasemonkey script to keyboard-navigate the Gmail simple HTML view. Very handy.

More and more sites use "Content Security Policy" and prevent javascript: from working.

Reply on news.ycombinator.com