Hacker News Reader

Comment by kris-nova | Hacker News Reader

We debugged a lot of the behavior using hackyderm.io as an HTTP facade for hachyderm.io with independent TLS termination.

I was able to get a single tweet in using hackyderm.io and my account is now “shadow banned”. Meaning I can view my tweet, but nobody else can. Which is exactly what Twitter has promised not to do, and what we use as a tactic on Reddit to keep shitty content at bay. It means that from the posters perspective everything looks “fine” and their content is just not getting any engagement.

https://twitter.com/krisnova/status/1603637253959733248

View on HN

Currently, whenever anyone posts a tweet including a URL, Twitterbot accesses that URL and if censored content is found (keywords associated with Mastodon) then the tweet is blocked.

It appears that this behavior is pretty simple and can be defeated by, for example, the following nginx configuration:

        server {
            if ($http_user_agent ~* "Twitterbot") {
                return 200 ElonIsGreat420TSLAToTheMoon;
            }
            return 301 https://mastodon.social$request_uri;
        }

If they start spoofing the UA one can also look at $server_protocol and if != HTTP/2.0 then do something different. I don't know what Twitter supports but most search engine and chat platform crawler bots can only speak HTTP/1.1. All the mainstream browsers support 2.0. This is assuming http/2.0 is enabled in the web server.

I doubt it's "as advanced" as doing keyword matches on the page in question, though they might well do some of that as well. I suspect the main reason for hitting the link is to resolve redirects.

I run a plain Mastodon install on https://m.galaxybound.com/ and I can post links to it just fine. If it can't even detect a standard Mastodon install, it's not a very successful search for blocked content.

It seems to me they're stupidly still maintaining a blacklist of the larger instances.

Note that I've "even" tweeted links to a post on my Mastodon instance that contained links to a blocked instance, and Twitter didn't even detect that.

A trick malware distributors use is adding a JavaScript-based time delay to their phishing pages. It's slightly more annoying for scanners to detect than just an UA switch.

Use TinyURL. Shortened links seem to work.

I use Friendica for posting to Twitter and if that happens my Twitter profile would be gone real quick. Now that’ll have me regularly checking my posts though lol

Reply on news.ycombinator.com