Twitter preconnects to the wrong domains
ctrl.blog
ctrl.blog
For the last weeks until a few days ago I've been unable to access twitter unless doing a hard refresh on every click. Because they installed a faulty service worker of some kind that would break most of the requests. (Both on desktop and android)
And on Android, every time I follow a link to Twitter in an app that opens in a web view, it gives me a faulty page that I have to refresh a few times before I'm able to view it. It loads the page fine, but some rest call or whatever to fetch the tweet crashes.
Edit: Not heard many others complain about this before, so mainly thought it was something about my setup. But the huge amount of upvotes this got suggests some Twitter engineers better look into this..
I get this issue too, I figured it was part of some trick to make me want to use the app.
The button to “Try Again” doesn’t work either, you have to manually reload the page.
This is in Apollo on iOS, but it sounds like the same broken behavior happens in web views on Android.
I had been under the impression that service workers required approval from the user to be installed. I had service workers from websites that I haven't visited in years and don't even exist anymore, sitting there chewing on my RAM every time I started Firefox.
Also Twitter has also been broken for me this way for months.
Service workers have some nice features, but I submit that they need to be something to be whitelisted by users, not something websites can just toss in there whenever they feel like it. The odds of one of them screwing up and eating far more resources than they have any right to approach 1 as the number of them increases. I don't know exactly how they managed to collectively eat 2GB of RAM and I don't care; there is no way they were bringing that much value to me, especially as I permit no website to simply push me notifications.
[1]: A suspiciously round number; it was exactly 200. Is that a limit? I don't see it in about:config.
Time to remove these. Thanks!
[0]: https://addons.mozilla.org/en-US/firefox/addon/cookie-autode...
If you want a cache, use HTTP caching, not custom JS code.
I ignored Twitter being broken for a bit since I didn't mind not using it. Then I realized my Firefox battery usage had essentially doubled and was draining my battery faster than normal. Once I cleared all Twitter site data including the service worker, back to normal.
Thought that my IP had landed on some list where Twitter degraded the connection and I wasn't even logged in.
Recently, it appears that I can't load twitter URLs with query parameters - the Guardian use some really weird ones of these, but when i delete them the URL loads. Super weird.
Happy to know that Twitter is just being Twitter.
Kinda funny that twitter can't figure out such basics such as opening a page correctly when following a link. This has been happening for months.
Web development is hard.
If I remember correctly there was a brief period a while ago when they did use a JS-only "web app", but then switched back to simple HTML with JS enhancements (and I seem to remember they announced that change with much pride).
Of course, there is always mobile.twitter.com that is still static-only and quite usable without JS, but IMHO Twitter is the perfect example of how the "modern web" is doing less with more.
So there's a persistent cache of javascript that can't be cleared except one at a time? My cache is full of fly-by-night websites. Surely this is a shocking privacy hole?
Web browsers need to be more transparent about what state they are storing, and more accommodating of attempts to clear it.
Also a performance/battery suck. Apple refuses to implement it in Safari, and it's one of the big reasons web developers deride Safari for being "the new IE"
That's far from obvious. Good cache control and offline usage are great for battery usage.
> Apple refuses to implement it in Safari
Wrong, Safari supports them. What Apple doesn't do well is enabling webapps that you pin to your homescreen ("PWAs") to work (they allow it, but from what I hear it's buggy and has weird restrictions), because they'd rather force you to make a native app.
In the wider concept of "PWA" (Progressive Web Apps), they also are used to enable webapps to act a bit like local apps if the user opts in - a user can add them to the homescreen, and they then are launched just like normal apps (but still sandboxed in the browser). An example that does this quite well is the Web-IRC client "The Lounge" - you can open and use it in the browser, but also pin it as a standalone app, including support for push notifications etc if you allow that.
I agree with the transparency, I was not aware about service workers until this post. In the meantime, there are Firefox/Chrome extensions that manages the service workers. Those extension can block service workers from installing without the user consent which that was nice.
AFAIK it's fine because they get cleared when you delete cookies/site data, so it's no worse than a site using localstorage, for instance.
Twitter is literally so buggy. Most of the time I have to refresh a page several times. Maybe it's because I don't have an account and refuse to use their apps?
I'm not sure whether it's because I've setup FF containers to open all twitter.com links in a separate container.
Or as @miffe said, "Go to about:serviceworkers and ctrl-f twitter and remove every instance. That will fix it."
Outbreak: index-sw-9a4c43b4b4778e7d1ca619eaaf5ac1db.js https://youtu.be/CPP9ew4Co0M
Kudos to the OP for investigating and documenting their findings.
@Twitter: please fix! kthxbai
Not ideal
I know most people don’t know the difference, and it would generally be a bad idea to have your www not redirect to the bare domain (or vice versa), but personally I prefer when we don’t hide these things. Just a bit of pedantic correctness, I guess.
> I can’t look at older versions of Twitter, as its pages don’t work well in the Internet Archive’s Wayback Machine.
Now this really gets to me.
While we’re on the topic, I wonder why some websites chose to redirect to www and some keep the apex domain as their homepage. I use the latter because it’s cleaner and, well, my domain is primarily used for my website, but I wonder if there is a technical reason behind choosing one or the other.
I don't think many people get seriously confused by it, but accessibility is for everyone and it just makes it that much easier for people to know what to do.
Just a guess.
* http://jdebp.uk./FGA/dns-srv-record-use-by-clients.html
Problems are more along the lines of conflicts with internal Active Directory deployment and "split horizon" DNS service.
* http://jdebp.uk./FGA/web-allowing-omission-of-www.html
* http://jdebp.uk./FGA/dns-split-horizon-common-server-names.h...
You can have a wacky DNS server that allows you to do weird things like ‘alias’ records, but those are sidestepping the DNS RFC: They’re answering with A records, but in a dynamic fashion.
I assume the RFC explains the reasoning, but prima facie, this seems like a bad change to me.
In that sense using www.basename.tld is thinking about (or at least autonomicly mitigating, by way of scope limiting) those potential security/privacy issues.
I don't think this is true. There was a time about 15 years ago when everyone fancy went without www but the pendulum has swung back completely in the meantime. Most sites nowadays redirect to the www version. Twitter is the only page I am aware of which was no-www from the very beginning and stuck to it consistently until today.
now everything is HTTPS (or tunneled over HTTPS) so it makes little sense to specify mundane things such as the machine you want to connect to and the protocol to use... there is only one machine (the apex domain) and only one protocol
There is no reason that imposes www.example.net to serve the same content as example.net. Whether or not it does is left to decide by the website operator.
That's why everything is moving to HTTPS endpoints. It's simple. You just give a url and it works.
Technical point: FTPS is FTP with TLS, but SFTP is a completely different protocol based on SSH.
The SFTP protocol itself, the thing spoken over SSH to the remote SFTP subsystem, is pretty simple, although I don't think it was ever formally standardised, https://tools.ietf.org/html/draft-ietf-secsh-filexfer-13
You could in principle talk SFTP over some transport other than SSH (e.g. you could use ALPN to select it over TLS), but nobody does.
network.predictor.preconnect-min-confidence
in Firefox.More likely it's for when FF is making a connection because it predicts that page is going to use a connection to another origin
From wireshark capture of `openssl s_client localhost:443`
2kb can add up if done often enough. And this is a fairly ideal case. I just have two simple certs using secp384r1.
If certificate chain is longer and when the certs use RSA, it will consume more bandwidth.
That might be why they don't add the subdomain, because adding a unique subdomain to track a user. is free and a domain isn't.
https://twitter.com/CodidactQA/status/1321237358776373248
https://i.imgur.com/cdh8rKZ.png
edit: prepending software.* to a domain prevents the subdomain from being removed, https://twitter.com/lukerehmann/status/1321310972468973568 I've also tried messaging this same format of unique link and the preconnect slides right into the DMs
"Technically, it only preconnects when you hover over the link."
The t.co link also helps them block URLs that they deem problematic on their platform - in the event of spam, attack, or abuse, the redirect can instead be a black hole.
[1] Note: Modern SMS apps support a protocol that sends larger messages as a stream of multiple interlinked SMS. This is transparent to the user, but at the time of Twitter's SMS interface this was not common.
I thought browsers requested a domain lookup (gethostbyname()) and basically got back a "zone file" which would have the cnames in. So, I was confused, when people complained about Twitter forcing a domain lookup "on the wrong domain" as I was assuming this would at least cache the domain lookup: It's the right domain, of course, but the lookup for an address on a subdomain includes the subdomain and then gets the cname directing wherever.
It always confused me that dig/nslookup didn't seem to provide all the info. They can, using nslookup 'ls example.com' or 'dig example.com -t AXFR' but the server in general refuses to serve the zone file (seemingly for security by obscurity reasons).
So, for example, if the browser looks up example.com it doesn't get that there is a cname from www->example.com . It only gets that relationship from looking up "www.example.com".
So, TIL, and now results provided by dig/nslookup on the command line make more sense!
* Google (maybe deliberately)
* "The almighty WHATWG" who accepts Google's revision on whatwg/url about this
Hah, what a dumb comment. Let's just go back to AOL keywords... but we can call it Google keywords.
I propose a new URL scheme: "web:nytimes/some/article/". Sadly I don't work for the Chrome team, so I can't just force it down the web's throat.
This is already what we have, except that "web" is "http" and we have TLDs to namespace domains.
We do have them, but as everyone has already realized, namespacing domains isn't really a good idea.
Could you expand on this?