I miss the programmable web (2021)
matt-rickard.com
matt-rickard.com
I miss: bookmarklets and user scripts
https://twitter.com/marekgibney/status/1551483561621979136
The bookmarklet takes me to the latest HN post I have not seen. When I click it again, it takes me to the next I have not seen. As soon as I see a post I have seen before, I know I am up to date.
I can't imagine reading HN any other way. Because otherwise, I would have to skim the whole front page to see what's new.
It also works on threads. So, when I post something on HN and want to see the latest replies, the same bookmarklet does this for me.
I have bookmarklets for every site I use frequently.
javascript:if (typeof e!=='undefined') e.style.background='#ccc';
e=Array.from(document.querySelectorAll('.age%20a:not(.s)')).sort().at(-1);
e.style.background='#ff0';e.classList.add('s');
e.scrollIntoView({behavior:"smooth",block:"center",});{}
javascript:e=Array.from(document.querySelectorAll('.age%20a:not(.s)')).sort().at(-1); if (typeof e!=='undefined') { e.style.background='#ccc'; e.style.background='#ff0'; e.classList.add('s'); e.scrollIntoView({behavior:"smooth",block:"center"}); }Here is a repo from which it is easy to copy:
The {} prevents the code from evaluating to a value that the browser then would use as the new value for the DOM, overwriting the existing page.
Try bookmarking javascript:123 and see what happens when you click it.
Bookmarklets are URLs, so a newline in the code would be urlencoded as %0A. So you could do:
javascript:alert(1)%0Aalert(2)
But I would prefer:
javascript:alert(1);alert(2)
javascript:if (!location.href.match('ycom')) {location.href='https://news.ycombinator.com'};if (typeof e!=='undefined') e.style.background='#c0c0c0';e=Array.from(document.querySelectorAll('.age%20a:not(.seen)')).sort().at(-1);e.style.background='#ff0';e.classList.add('seen');e.scrollIntoView({behavior:"smooth",block:"center",inline:"nearest"});{}
And accessing the variable e before assigning it a value, now I see that probably serves a practical purpose too.
haven't checked in a while though, maybe there are workarounds
Make all the data free to use for everyone, while ML techniques get even more accessible, and let people build products with that - where they also have to provide APIs for their enriched data.
javascript:(function() { (document.querySelector('video') || document.querySelector('audio')).playbackRate = (+prompt('How many times normal speed?') || 1); })();It's baked into YouTube Vanced on mobile, which was sadly taken down by Google. Smart Tube Next is another Android app with this feature, but I'm not sure if it's available for anything other than TV set top boxes.
It used to be browser extensions were only used by a select few power users, if you don't count those installed by malware/shitware. Now it's common to see folks who only use their computers for Instagram and Youtube having all sorts of browser extensions installed intentionally, and use regularly. And they're essentially just bookmarklet/userscript bundles with access to some special extra APIs.
What's more, with the proliferation of Electron in desktop applications, now even more of the UIs we use are programmable.
Some of this navigation and some of this content was in VRML that you could interact with. The full product tour could be explored in a 3D world. The UI UX did not seem OTT. It felt like the future.
It turned out that the excellent CosmoWorlds VRML authorship tools were just a little bit ahead of their time and there was little of the expected interest. We ended up with a flat 2D web rather than a 3D web. Sure this makes sense but the tools you have available frame how you solve problems.
Also out the box was a web server. You put HTML (and VRML) in your shared web folder and your machine instantly had your stuff available on the local network and you could route to the machine from the internet if you had the networking skills to set that up.
It was all in the original Tim Berners Lee vision, you could imagine how this would make sense in academia, with academics in a department able to put all their work/knowledge on the web very easily.
In this SGI version of the 3D web it was the javascript that glued it all together, VRML was made useful with scripting which could go across those frames to put together an awesome web experience, rose tinted spectacles accepted.
We aren't even thinking along those lines these days. The web has gone, we have platforms now.
We aren't even thinking along those lines these days.
Apple's been driving a "model" element for the web. Personally Im unthrilled that it's an object but bereft of an environment; there s no real space or navigability.
On a site note: Been looking for one to show reccomendations always based on simmiliar vids, not my recently watched and whatever the black box algorithm is offering me. Unfortunately I never figured out how do it properly, since the position of the element seems to be random and sometimes straight up missing.
1. Prevents channel page video from auto-playing (not mine)
2. Removes the stupid custom scrollbar that takes up actual pixels on my screen at all times
3. Automatically sets highest available video quality by clicking through the quality menu so I don't have to
And those are in addition to SponsorBlock and uBlock Origin extensions.
www.google.com##div[role="heading"]:has-text(/^People also ask$/):upward(5)
www.google.com##span:has-text(/^People also ask$/):upward(5)
www.google.com##div[role="heading"]:has-text(/^Cast$/):upward(5)
www.google.com##span:has-text(/^Cast$/):upward(5)
www.google.com##h3:has-text(/^Videos/):upward(5)
www.google.com##h3:has-text(/^Images/):upward(5)
www.google.com##div[role="heading"]:has-text(/^People/):upward(5)
www.google.com##span:has-text(/^People/):upward(5)
www.google.com##h4:has-text(/^People/):upward(5)
www.google.com##span:has-text(Related searches):upward(5)
www.google.com##span:has-text(/^Latest/):upward(4)
www.google.com##h3:has-text(/^Top stories/):upward(6)
www.google.com##div:has-text(/^Trailers/):upward(4)
www.google.com##g-scrolling-carousel% curl -s https://manifold.markets | htmlq '#__next > div > main > div.items-center.flex.flex-col > div > div.gap-8.flex.flex-col > ul > div > div > div > div.relative.flex-1.gap-3.pr-1.flex.flex-col > div.peer.absolute.-left-6.-top-4.-bottom-4.right-0.z-10 > a'
will get you all the anchors for the stories on the home page, and from there it's easy to get any other part of the stories.
But tbh I can't really imagine any scenario that is near impossible - except for WASM driven web pages which will break just about anything :(
Flash was at least easy to decompile. The decompiled code was so good most of the time that you could compile it back into a working swf, with modifications if you want any.
But then both WASM and Flash have to get the data from somewhere. That is usually something resembling an API endpoint that returns the result in XML or JSON. Why not sidestep the whole client app thing and go straight for that API endpoint?
BTW: were you around in the days of Flash? Because it broke accessibility and a whole host of other features the same way, yet it was still used and hugely popular (there was also ActiveX, which worsened things even more). It began with online games and graphics-rich pages, but it evolved frameworks for generic apps (ie. Flex) and then even CMS engines. Soon enough Flash was used for even blogs and e-commerce. The situation today is different due to the Web having evolved into a more capable and standardized platform, but there's no shortage of devs who would welcome the ability to run their app in the browser, but who dislike HTML/JS so much they would do basically anything just to not have to touch it - accessibility and a lot of other things be damned.
Don't underestimate how irrational people and even whole communities can be.
As a bonus, here's an example of the "almost" part: http://35.158.218.205/experiments/webDOOM/
Two things make this vastly more difficult now:
1. Twitter uses obfuscated css classes that appear to change on every deploy, which makes scraping hard.
2. You used to be able to get around the limit on bookmarklet code size by adding a script tag to the page and loading remote javascript. Now browsers respect HTTP headers that sites can use so that only scripts from whitelisted sites are loaded.
I could see using a set of standard tweets you created to sort of benchmark the change.
It was eventually abandoned.
You'd probably think it'd be simple, right? This being the kind of thing that Cascading Style Sheets were designed to handle elegantly and gracefully?
https://maya.land/user-styles/youtube/
I always thought people on here were just cranks complaining about style bloat, because a lot of the "this could have been a text file" arguments were applied to things that were genuinely better with the presentation polish. But this? How can developer ergonomics be worth this?
Anyway, user styles and user scripts and bookmarklets are all great and I use them all the time. This level of Big Tech deformation of the basic web technologies makes them more annoying, but still not impossible.
It's still around: https://patorjk.com/blog/software/
I wonder why no one ever made something like this for the web.
Userscripts will run, but if they try to load any content from elsewhere, that breaks. So, for example, the Hypothes.is extension wont load. Affected sites keep growing. https://twitter.com turned on CSP many years ago.
I agree. This is simple & stupid a problem. It's grossly anti-user. But today, security rules, and security sees all the misdeeds of users, and those who would prey on them, and they have no emapathy left for creativity and fun. There's no moral-value-party left anywhere that defends user freedom, that is pro end-user-hacking.
The Manifest V3 (MV3) catastrophe is still scheduled for 4 months out, even though there is still only 1/3rd the work done thay it would take to allow GreaseMonkeyTamperMonkey/et-cetera to live. The new browsers are actively unwinding power & possibility, forever trying to undo the permissiveness that it had (ex: CORB, and now ORB). The standards authors & browser makers are infected with deep deep fear. Certainty & closedness is winning.
Another way to say this is that we have web developers now when the original web was for web authors.
"One of the fundamental features of CSS is that style sheets cascade; authors can attach a preferred style sheet, while the reader may have a personal style sheet to adjust for human or technological handicaps."
So CSS was originally intended for both users and web devs, but web devs "hijacked" it to be for them alone - ultimately, by removing support for user stylesheets in the dominant web browser.
By "no-competition" I mean for the professionals/employees. There's so much demand for programmers and so much money allocated for them that the limited supply makes it trivial and common to be a professional today with skills on the level of a script-kiddie from the '90s.
Grease monkey is still alive, I added voice recording and speech to text transcription to Slack before they did.
For almost twenty years I'm thinking of expanding these two bookmarklets to use the rel=next/previous link relations instead, but I don't have high hopes in the quality of the markup of the web anymore. Which is a small problem for the idea of a programmable web. The deeply nested div trees which often result of modern frontend development and weird CSS methodologies are as worse as the table layouts of yore.
Strong disagree. Extensions and User scripts are an unmapped wild west.
Providing a registry type system to securely vend packages/scripts is a very hard security problem.
https://www.bleepingcomputer.com/news/security/mozilla-block...
It added a substantial amount of complexity but it wasn't insurmountable. The scripts he talks about sound like there's stuff I'd have to sub in a shell script or use a two stage process to do, as there is some post-download processing I'd like to do automatically, but there's no way for me to access my OS from the sandbox of Firefox (I mean, for good reason, although it would be nice if I could override that in some cases)
I am personally aligned with pro-CSP because it can greatly reduce the attack surface of web sites I host, and is quite effective and precise.
Browser extensions can, and do, play around CSP.
Bookmarklets has no overhead in the browser, because they are just bookmarks that do not interact with the browser unless I click it. Having a little addon for each little functionality is annoying, slow, and difficult to maintain and review.
Bookmarklets and user scripts are collateral damage.
The browser remains in a locked down kiddie mode until you solve these riddles three!
Go to discord.com (no need for account) and open browser console.
You will see
> Hold Up!
> If someone told you to copy/paste something here you have an 11/10 chance you're being scammed.
> Pasting anything in here could give attackers access to your Discord account.
presumably they added it after it kept happening. And likely the same things happens when random user can somehow run code they got from scammer :(
CSP literally says, only talks to these specific domains. https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP#exampl...
The browser doesnt block thr bookmarklet from running (iirc). But most bookmarklets immediately try to download & run some libraries to do their thing. Or they try to send data somewhere. And CSP is blocking those connections.
Its hard form me to guess how much of this is intentional anti-user lock out, and how much is just oversight or technical diffifulty. I could see not wanting to drill backdoors through your own security policy, which is more or less what it woild take, but it sure feels lile a loss. Users can turn off csp for the browser with a flag if they want, but we can only do that to the whole browser, not site by site. Also worth pointing out that just disabling csp for thr browser/site (rather than carving exceptions for userscripts/bookmarklets) is something sites can detect (by succeeding in a request that ought not go through) & could potentially decide not to serve you content or what not if they wanted to be petty. That said, extensions exist & arent readily detectable so there's options... they are just nowhere near as direct to author & use.
I wanna give a shoutout to Stylus[0] for offering userstyles (basically the same thing as TamperMonkey but for .css files)
It never occurred to me to try, but I think this is still possible even without the javascript: URI scheme, as there’d be nothing stopping you from putting an inline <script> in an HTML page described by a data: URI, and bookmarking said data: URI.
Not sure what the browser would consider that data: page’s HTTP origin to be for cookie / CORS purposes, though.
Tree-based history is smart, so smart that I don’t understand why no « mainstream » browser implemented it yet.
Qutebrowser is my favourite among them.
• https://news.ycombinator.com/item?id=27695463 (2021-07, 96 comments)
• https://news.ycombinator.com/item?id=18830430 (2019-01, 47 comments)
Now decisions are taken by managers or PO who don't care (or even know) about programmability, open standards, etc.
Simple web is better.
I get the utility but I suspect writing greasemonkey scripts for SPAs might not be a great experience.
Now it's install node, install git, install npm, install a gazillion packages, run localhost then hope everything works. Build command, clean command, eject commands, deploy command, pre processors, post processers, documentations full of so much jargon that i feel burnt out looking at it, hydration rehydration, pre-rendering, headless cms....on and on.
I find myself reverting to writing simple bash scripts, userscripts - they are more enduring, not built on a stack of fickle dependencies, fads and ever increasing complexity.
You lose the nice stuff like livereload but you can hack in something to that effect if you want to.
if it was up to me i would never touch those things, but that's what the market has become.
i even resisted learning typescript for so many years, because i loathe the idea of learning a language that needs to be converted to have correct syntax. same with jsx, i use it, but it feels horrible.
not everyone is willing to give the benefit of the doubt.
This is absurd. I write code for a living, I enjoy solving problems and that is part of my job.
I do not write code in my free time. I play video games, go hiking or go to the beach and maybe read a book with dragons in.
That doesn't make me a problem. My job is not my life. Honestly it shouldn't be, it is not healthy imo
I prefer candidates who show that they understand their fundamentals, their primary domain and can show that they grow and adapt easily. The current hot technology is ephemeral and can be learned on the job by a competent engineer.
That said, my background is startups so I don’t know what larger companies expect.
These days it feels like everyone is constantly looking for an angle, feeling like nothing is worth doing if it can't be monetized and turned into content. Fuck's sake, people can't even go _fishing_ without strapping up with a GoPro and trying to sell their favorite tackle.
Be punk rock. Go do something for the joy of it. Go learn a language that no one will ever pay you to write. You have a day job to get paid. Why worry about it for what you do for fun?
you're right, i'm trying to get into that mindset. Being contrary for no other reason than absurdity.
If I interviewed a developer and they showed me an impressive website/application that was built without a framework, I would probably be more impressed than if they showed me one that was built with one. An understanding of the basic building blocks of the platform is always helpful, just like I would consider knowledge of lower-level programming and different programming paradigms a huge plus, compared to someone who's only ever done web.
I would, of course, also be sure to ask them about why starting from Adam & Eve is only rarely appropriate in a business context, how it's important to write code that others can pick up and understand. I expect anyone but the most junior candidate to understand that no tech stack is perfect, and that it's all about trade-offs, and that the tech is a means to an end. We don't hire people to love a tech stack (most of them are pretty unlovable, to be frank), we hire them to, as quickly as possible, get good software in front of paying users.
If you come in and demonstrate that you can take a thorough understanding of HTML, Javascript and CSS up to something resembling a dynamic, modern, responsive web application, and if we can have an intelligent discussion about what the frameworks and infrastructure bring to the table, you're obviously miles ahead of someone whose knowledge stops at copy-pasting React snippets and googling the WebPack error.
Of course, having to work with multiple devs on the same project adds lots of complexity to minimize risks and conflicts.
That's when you link a GitHub repo to Netlify.
...I genuinely wondered what tree was involved. Took a Google search before I understood that it was industry jargon, which meant this: to reduce/minify code.
There are many similarly opaque things in the world of web-development.
Much of this was because real compression methods like zip simply weren't available during early decades of web development.
Obviously, if shrinking the code size is desirable, and you were building a library that did minification, you'd want to move on to also adding other, more dangerous changes that could change the meaning of the code, like removing unused CSS classes, doing DCE on javascript, etc, etc.
--
Since programming is a giant, decentralized soup of autodidacts, there's no "word of god" authority that can really say "this is the one true name of something". It's mostly just lingo that passes in and out of various communities, and a lot of times communities (as you're seeing in splintercell's comment) try to helpfully match themselves up and standardize so it doesn't degenerate into pure chaos.
Every industry has things that have names that you might need to learn.
Basically remove everything not connected to the root.
If I had a vote on the matter, it would still be fully SSR with a little bit of vanilaJS when need.
I likte to think about the the term development UX in contrast of end-user/mass UXs.
Every dependency is a benefit and a liability. It's been interesting to watch this ecosystem grow over the past decade as we collectively explore how much liability we're willing to accept.
If what you are doing is just another CRUD-based website, there is real need for more, PHP is designed to make that simple. I had a well deserved reputation for poor security, but they got better, and 90% of it is addressed by just using parameterized SQL queries.
It used to be you could open up your dev tools and figure out how things were made. And you could learn all sorts of things about architecture and style and techniques this way.
Now its all buried in build processes and obfuscation techniques, as if your button handler is a tightly kept secret that's somehow protected by scrambling your variable names.
Although the other day I came across a complex app that was built in modern vanilla JS with native ES6 imports alongside regular old CSS. It was a work of art.