Hacker News Reader

Comment by m3kw9 | Hacker News Reader

Just by looking at how easily their entire db was downloaded, wouldn’t be surprise they hacked most things together and hardcoded a lot of stuff to work with AWS

View on HN

If I recall, that wasn't the DB of Parler, but of their wordpress blog.

It's still sloppy, but it's not on the same level.

IIRC, one of the poor design choices was making their resource URLs (videos, pictures, etc) sequential - this reduces the effort of crawling everything down to a 'for' loop; no need to get the actual "db" for references. As a bonus, those media resources were not stripped of metadata (EXIF data in images was present)

And if it was indeed 70 TB downloaded and they were serving it off AWS, they're getting at least a $5600 bill as a last fuck you on their way out, just for that single download.

If you have enemies on the Internet, having anything on one of the cloud providers that charge an arm and a leg for traffic seems like asking for a financial DDoS.

There is nothing wrong with this on its own. The real flaw was not stripping the exif data. Although I guess sequential IDs did enable an easier attack on the real flaw.

... like this website? ;P I honestly don't see anything wrong with that design: permissions should be honored--and Parler did something really wrong here that caused people to be able to gain admin status from them--but, past that, if someone wants to download the whole site why not let them?

This website doesn't do private messages an such. Every comment and post is public. However, on social networks where you do have that function, a user might have a legit reason to want a video to be private.

When you have media that is private, if the numbering is sequential, it is easy to figure out what you can't see vs what doesn't exist. Using random IDs means that someone looking from the outside cannot reliably figure out how much content is hosted and check for private data on that service.

A secondary issue is that having an auto-increment in your database is a recipe for having really slow HA options and you can't scale hard either because it's fundamentally hard to sync auto-increments between multiple nodes that don't leave gaps in the ID namespace or are really slow.

Knowing how much data is private isn't a useful quantity? If it were per-user I would agree. Honestly, even knowing "a private video was posted at such and such time" for a website like Twitter isn't a big deal (and this is good, because Twitter also uses an auto-increment key for all of their posts, or at least used to until not that long ago if somehow they recently stopped).

(FWIW, Facebook actually used auto-increment keys for a long long time for their users. I don't remember when they stopped, but before they did they went through a long period of manual ID range allocation.)

Twitter uses Snowflakes, simialr to FB and other services. They scale better than manual ID range allocation and don't entirely behave like auto-increment. The big part is that it can contain gaps. So there is no way to tell if an ID doesn't exist because you lack access or because it was skipped over.

Reply on news.ycombinator.com