Intercepting t.co links using DNS rewrites
djharper.dev
djharper.dev
But this is part of a wider pattern of the internet becoming deep fried, instead of a link we get a short code to a facebook page with a bot reposting tiktok videos of a phone screen-recording of an editorialized livestream of somebody watching a screen-recording of the video on youtube - and ofcourse it starts half way through then loops round again and plays twice with three sets of black bands, content creator @names, watermarks, wifi & signal indicators etc.
If we could all do one thing to help stop the spread of this cancer, that would be great: de-duplication via content addressable links / tags.
But it'll never happen.
Good phrase. I've frequently heard the term "Kentucky Fried Education" applied to the decay of schools and universities under the influence of big-tech mediocrity. My Digital Vegan response would be that "heavily processed" content harms your intellectual health because it's not fresh and has no vitamins :)
Ted Nelson anticipated this "death by indirection". Some of his early writings on hypertext still seem like they're from a future we might one day get to. The idea that links should be bi-directional, and how different the Internet would be, is still mind-blowing.
Most of the problems you describe stem from the mediating platforms being fundamentally controlling and dishonest. Everything is a work-around to around a work-around... all the way down.
> will never happen
I get the shrugging cynicism. But to me it's a good thing they're too broken to change. Let these festering sewers of low-quality content digest themselves and make room for new growth.
It works well! I'm running it with Nginx in a Proxmox LXC container where I've allocated a meager 64MB of RAM for it and it still has RAM to spare. I wish I could say the same of other "small" tools from across the web that I'm running.
I like the minimalist web pages and the fact it auto-resolves multiple redirects on its own (bit.ly -> msft.it -> aka.ms -> ...) without making you wait for the page to load and the fact it removes tracking parameters for you. I know there are online tools and extensions that the same but those are a pain to install on mobile.
But still, glad it worked!
1. Make all shortening services append a `This-is-a-shortening-service: true` header to all the responses they send.
2. When a link is added to a shortening service, check if the response from the link has the header above and resolve the destination, recursively.
/? shorturl api OpenAPI https://www.google.com/search?q=shorturl+api+openapi
- TinyURL OpenAPI: https://tinyurl.com/app/dev
- GH topic: url-shortener: https://github.com/topics/url-shortener
A https://schema.org/Thing may have zero or more https://schema.org/url and/or https://schema.org/identifier ; and then first the ?s subject URI that's specified with the `@id` property in JSONLD RDF.
You can add string, schema:Thing, or URI tags/labels with the https://schema.org/about property.
I understand why both HPKP and Expect-CT have been obsoleted, but it's a bummer that we still don't have a good enforcement mechanism for CA/cert pinning for a particular site. CT itself does a reasonable job of mitigating the "globally visible mis-issuring CA" problem, but does nothing to help users whose certificate stores contain all kinds of mystery enterprise or application-installed CAs.
Not necessarily; the argument is that it's indistinguishable from a malicious MiTM. I think this is a great and legitimate use, but it's also probably something that website providers should be able to make themselves resilient against (or, at the least, be able to audit when it happens).
I think there's a reasonable argument to be made that it's in the website's (and my!) interests to be able to detect and prevent these kinds of man-in-the-middling.
It doesn't matter if it's in the websites interests. The client computer does not belong to them, and it's definitely not in the owner's interests to let others "audit" them just like it's not in web hosts interests to let us "audit" their nginx configs.
When I say "audit," I mean in the sense that existing ecosystems like CT already provide automatic auditability of certificate issuance. We're not talking about a private company sleuthing through your computer; we're talking about a way to enforce the stated security model that most users expect when a connection is described as "encrypted."
So there is that issue....I guess one way to mitigate it would be to run the sidecar out of the network, or at least have a clean DNS config and not have my custom CA in the root store...i.e. you'd want to be double sure you're going to the real thing and only accepting trusted certs signed by a trusted root.
I thought it was odd you kept calling it "bad" and "awful". It's exactly as secure as any other certificate on the web. Arguably more, as you're aware of any access to the keys. The only differentiation is a commercial interest, nothing "bad" at all.
I wonder if there would be some way to redirect kinda even if you weren't on your home network... You mention using a VPN? Which one?
The annoying part is having to copy the link, navigate to the website, paste in the link etc.
I was looking for something more "seamless" and works cross device (e.g. on phone, in the Twitter app etc) not just browsers. With this you just click the t.co link and the result is there instantly.
It's a dumb solution but was fun to write.
Not a caddy user, but I believe caddy, with a plugin, can function as a forward proxy.
Mission accomplished!
Additionally if I want to share a URL with friends it's usually polluted with all sorts of nonsense query parameters, this tool strips them and gives you a nice clean one.
As for the tracking element yeah, the link shortening services will still see my IP connecting to the service, doing a head request, just with a different user agent (whatever the one go's stdlib sets). One extra layer could be to move the go service out of the network, but tbh I didn't start this project with these considerations in mind
It only resolves the links navigated to, right? It shows you the interstitial page, but only when you try to navigate to the link.
It does enable you to get the clean link and share that.
Browsers could in theory contribute data but the infrastructure to support that would likely to be orders of magnitude bigger. I'm scared even thinking about going down that rabbit hole due to the expectation of what would be found (MITM evidence). :_(
There's no need to track certificates from private CAs. In order for your browser to trust the certificates you need to manually trust the CA (or your system admin will configure it). Everyone else will see it as invalid.
List of supported ones is here: https://github.com/djhworld/theunwrapper/blob/main/config/un...