So this guy is now S3. All of S3

To clarify - it felt like this was an attempt to replicate the mid-00s play of building interest by restricting who can join and making it exclusive (Facebook did this by rolling it out uni-by-uni, Gmail was for a while invite-only and invites were highly valued) and therefore desirable.

Maybe that's in my head but layering this feeling on top of BlueSky being yet another microblogging service with a few other things that I don't love contributes to my impression of Bluesky being simply "meh". If it becomes the next thing that everyone uses, I'll inevitably have to check it out, I'm not going to be an early adopter though.

So the thing is that in one respect they actually do get caching, almost to a fault. One of the complaints I've seen among some Mastodon instance operators is that they end up storing some pretty hefty amounts of data locally as their instance caches remote posts, images and profiles from other instances that its members follow. One source of problems, which may have been resolved, was that even though there's a job that cleans out this cache the banner images from external profiles stick around. I saw this a while back and it seems like an easy fix so I imagine it's been addressed.

I don't think I am equipped to diagnose what the root cause was here. It's even possible that this instance wasn't intended to have viral posts (i.e. high profile posts that get would get shared to many external users) and they didn't want to invest in hardware/services to facilitate this.

Thanks, will do! I'm always on the lookout for interesting people on Mastodon!

> Are we just relying on .well-known having some extra protection [...] ? If so then .well-known is little safer than any other arbitrary location in this respect.

If .well-known had just been invented, that would be true. It's fairly well established at this point, though. For example, if someone can create arbitrary files in .well-known, they are also able to pass http-01 ACME challenges and thus issue TLS certs for your domain (modulo CAA) and MITM you. At this point, allowing users to modify .well-known is about as good an idea as allowing them to receive mail for postmaster@ or accepting incoming packets from 192.168.0.0/16 into your LAN.

Amazon S3 specifically would not be vulnerable because bucket names can’t have dots in them; same for every other service that doesn’t allow those. Neither would services that prefix usernames with ~ or @ or similar, nor services that already use http-01 ACME challenges to get certs thus are already using that path.

I’d be much happier if proving domain control were only done through DNS challenges, but that ship has sailed.

The aforementioned trick wouldn't work if it wasn't. But before the 429 error was put into place the entire site was affected across the board by Hacker News. See https://chaos.social/@ordnung/110312020977014678 .

Mastodon is written in Ruby on Rails and there are some inherent performance issues with that, it generates a huge number of Sidekiq jobs that can bog down a server quite easily. There are other, non-Ruby implementations aiming for compatibility with the Mastodon API though, so I’m curious to see how it will all shake out.

Yikes, why didn't they use a /.well-known/ address instead of inventing a new directory? This is entirely on Bluesky, not AWS.

Stunning that there are (were) any 4-char bucket names left.

There's enough imperfection in the world that no conspiracy is required.

Microsoft funded Apple to keep another OS vendor alive, so it's not about discouragement. It's probably a lot cheaper to fund a competitor than paying the gov't and getting tagged as a recognized monopoly

Here's how I think about it:

* ActivityPub -> AT Protocol (https://atproto.com/)

* Mastadon -> Bluesky (https://blueskyweb.xyz/)

Right now, federation is not turned on for the Bluesky instance.

There are differences in both, however. I'm not going to speak about my impressions of the Mastadon vs Bluesky teams because frankly, Mastadon never really caught on with me, so they're probably biased. ('they' being my impressions, that is, I just realized that may be ambiguous.)

At the protocol level, I haven't implemented ActivityPub in a decade, so I'm a bit behind developments there personally, but the mental model for AT Protocol is best analogized as git, honestly. Users have a PDS, a personal data server, that is identified by a domain, and signed. The location of the PDS does not have to match the domain, enabling you to do what you see here: a user with a domain as their handle, yet all the PDS data is stored on bluesky's servers. You can make a backup of your data at any time, and move your PDS somewhere else with ease (again, once federation is actually implemented, the path there is straightforward though). This is analogous to how you have a git repository locally, and on GitHub, and you point people at the GitHub, but say you decide you hate GitHub, and move to GitLab: you just upload your git repo there, and you're good. Same thing, except since identity is on your own domain, you don't even need to do a redirect, everything Just Works.

This analogy is also fruitful for understanding current limitations: "delete a post" is kind of like "git revert" currently: that is, it's a logical deletion, not an actual deletion. Enabling that ("git rebase") is currently underway. Private messaging does not yet exist.

Anyway if you want to know more the high-level aspects of the docs are very good. Like shockingly so. https://atproto.com/guides/overview They fall down a bit once you get into the details, but stuff is still changing and the team has 10,000 things to do, so it's understandable.

This is a private beta. Nobody is suggesting that any of this be used for anything serious just yet. Development happens out in the open, you can go find out what else they've missed by doing the work, or by waiting until others you trust have done so.

I myself have had an account for like a month now, but only started really using it a week ago, because that calculus changed for me, personally.

Like, it's not even possible to truly delete posts at the moment. This all needs to be treated as a playground until things mature.

This isn't even the first "scandal" related to this feature already!!!! There is another hole in what currently exists that allowed someone to temporarily impersonate a Japanese magazine a few weeks back.

For me, the worst thing about it is that they didn't just use webfinger. So webfinger isn't perfect, but it's there and in use. When they choose to invent new mechanisms for things there are perfectly serviceable options for, it makes me instantly sceptical of the rest.

Wouldn't be funny if it was a public beta that they want people to use for serious stuff. But it's neither serious, a beta or public, but basically a private alpha for playing around, so i'd be a bit lenient on screwups.

Oof. This will not be the last time that decision causes a problem.

There's an account bucket limit, so you'd need to create a huge number of AWS accounts with no immediate benefit.

So to answer GP's question, yes.

If you go to the user's profile and then to the post, it seems to be okay. So perhaps also looking at Referrer.

Or running efficiently never maybe?

unfortunately this is exactly why mastodon won't work

I already don't trust mastodon links because 9 times out of 10 they simply don't work. Everyone's tiny hobby server falls over when one post gets big, and obviously not everyone is going to scale their servers to support the load of a viral post that might happen once every 6 months and will be 100x their base load

And the original tooter is apparently Google Translate?

Isn't HN pretty small? This post has <400 upvotes over 3 hours. There can't be 1000x that amount of lurkers can there?

Mastodon user count has mostly been a steady growth. So far it hasn't really failed at a high level. We aren't in Bitcoin territory, the network isn't really slower than it was a year ago even if number of user is much higher. It's mostly distributed among many instances.

Twitter is pretty sluggish, to be fair. 7 seconds to load and render a single tweet on mobile.

https://pagespeed.web.dev/analysis/https-twitter-com-realDon...

Mastodon.social is actually much faster on this particular benchmark. So maybe there is hope.

That just redirected me to too many requests.

> slowing down the entire site

This is mind-blowing. Last I checked, the front page of HN sends tens of requests per second to each link. There are humans who can pack envelopes faster than the typical mastodon server can answer GETs. I'd love to see someone benchmark the top servers for a few seconds to see what it takes to break a reasonable latency SLA.

Sites like HN and that other one should track this. It could work like flagging - if enough people mark it as hugged to death (HTD), it would say so next to the link. Maybe it could even redirect to an archive if it’s currently HTD.

lightly blew in its general direction to death

Neither of these are 'authentication'

You're thinking of how Mastodon does verified links. You could do something similar, provide a verified link on your profile to a file in an S3 bucket, but there's very utility (or risk) in that.

Mastodon also allows you to be discoverable via a custom domain, using .well-known as parent mentioned https://docs.joinmastodon.org/spec/webfinger/ https://www.hanselman.com/blog/use-your-own-user-domain-for-...

The /.well-known/ path prefix is the standard name to use (https://www.rfc-editor.org/rfc/rfc8615) so that any sort of “we’ll host user content from our domain” thing can block it. (Hosting user content from the user’s domain is fine and doesn’t need this restriction.)

A few things are effectively grandfathered in due to their vintage: /favicon.ico, /sitemap.xml and /robots.txt are the three that occur to me—so if you’re running something vaguely like S3, you’ll want to make sure users can’t create files at the top level of your domain matching at least those names.

But nothing new should use anything other than /.well-known/ for domain-scoped stuff, or else you run into exactly this problem.

That is basically the idea of .well-known

Webfinger is when you want to multiplex multiple identities on a single domain

E.g. https://example.com/.well-known/webfinger?resource=nick@exam...

Will serve the challenge proving your handle is @nick@example.com

.well-known is basically the same idea https://datatracker.ietf.org/doc/html/rfc5785

Or why not just serve it from www.domain.com/.well-known so we only have one thing to block. :p

s3 supports my-bucket.s3-us-west-2.amazonaws.com style URLs as well

Yes, but you don't have access to s3.amazonaws.com/.well-known, only to yourdomain.s3.amazonaws.com/.well-known.

> .well-known seems unintuitive

We're talking about folks setting up a custom domain for a personal social media presence. If they can handle nameservers and DNS records, they can handle a folder with a dot in the name.

> this really belongs in DNS.

And the primary way of identifying yourself is in fact DNS.

> I’ve been working on a solution to this

Your solution is almost identical to the BlueSky one: put a TXT record at _atproto.<domain> that resolves to a DID. The difference is that they mandate the DID spec and you do not. Which is totally fine! Just figured I'd let you know :)

I worked on a product that required DNS changes to set up. Especially for business accounts, the level of friction was STUNNING. We had it take months to get set up because the contact had to submit a ticket to IT, write up the business justification, get director level approval, get security approval, and so on before it could get done. We had customers who couldn't even figure out which group in their company managed DNS. Yeah, you can argue that those companies are broken, but as an outsider I have no influence over that. The result was just that they couldn't use our product. On the flip side, we had consumer and small business customers who had purchased domains through simple webhosting things that didn't give them the required level of access to create a record (and/or they couldn't figure out how to do it). We eventually added an HTTP option and the success rate and time to success both improved hugely.

Both http and dns verification are stupid. Neither of them prove you own the domain.

http verification proves you temporarily control IP space relative to a viewer. dns verification proves you temporarily control name resolution relative to a viewer.

Both are trivially hacked, multiple ways. By the time someone finds out you did it (if they closely monitor CT logs, which nobody does) you've already had hours, days, weeks to run a MITM on any domain you want. The attack only has to work once, on any of 130+ CAs.

The solution is registrar-level proof. Cert request signed by the private key of the domain owner, sent to the registrar to verify, the registrar signs it if its true, it's sent to the CA who can see the registrar signed it. Now you know for a fact the domain owner asked for the cert. The only possible attack is to steal all three of the owner's private key, the registrar's private key, and a CA's private key.

I have been shouting about this for 10 years, none of the industry incumbents care. The internet is run by morons.

> If you want to really prove someone owns a domain, make them change an authoritative DNS record.

You're not wrong (ignoring how easy it is to hack DNS), but at the same time it's hard enough to get people to buy their own domain name, nevermind understand the system well enough to add a TXT record.

It's a strategy that's fine to implement when your target audience is server admins. It's a terrible strategy when your target audience is everyday users who you hope own their own domain. Doubly so in a world where owning your own domain is so rare for individuals.

Convenience. DNS is routinely not automatable by API, or inconvenient to automate. HTTP, however, is normally easy to work with.

Sure, adding a TXT record to verify domain ownership is fairly common and lots of tools still use it. But you either have to self host DNS (yet another container to maintain) or use your provider's API (yet another credential, yet another mailing list to subscribe to for inevitable breaking changes to the API).

In contrast, HTTP based verification often has built-in support with your webserver (Caddy) or only requires copy-pasting a few lines to your docker compose file.

There are edge cases, but they're also widely exploited so you won't run into them if you follow best practices.

It's ridiculous that this is not in the title.

>Chaz uploaded his custom file to their Amazon s3 bucket

their is who exactly? and why does bsky associate it with the s3 domain if it's just a file in a random bucket?

I would argue this is worse (and more hilarious) than Musk buying and giving out checkmarks for people.