Hacker News Reader

Comment by candiddevmike | Hacker News Reader

> Usernames are replaced with consistent pseudonyms generated by the mimesis library, ensuring that identifiers remain unique and contextually meaningful across records. Similarly, user IDs and message IDs are hashed using the SHA-256 algorithm and truncated to 12 characters. This deterministic hashing approach maintains linkage between related records while effectively masking the original identifiers. The global name field, deemed unnecessary for analysis, is entirely removed. Additionally, user IDs embedded within the content field are identified via regular expressions and replaced with their corresponding hash values.

Seems pretty thorough, though this is may end up being a good lesson for GenZ/A not to post things in public spaces on the internet.

View on HN

So if I can identify a chat, that I have direct access to, in the dataset, I can get the hashed user ID of my contacts, and the search for any other messages from them?

If they didn't salt it, you might not even need to identify the chat - just hash the username you want to look up. I'd check, but it's a >100GB download.

I'd check, but it's a >100GB download.

The page given by pavel_lishin above includes a sample data set that's only 6.2 GB:

https://zenodo.org/records/15170676/files/dataset_sample.zst...

Got a link to the full dataset?

seems like. or if your chat mentioned a person IRL and not a discord username. or used a nickname.

> public spaces on the internet

But discord servers aren't considered "public spaces", hence the concept of an "invite".

This is akin to someone revealing they've been going to private parties and secretly recording everything.

It might not be illegal, but it's definitely not polite.

These servers do seem to be public, in the sense that anyone can join without explicitly being invited.

It sounds more like they went to the mall, picked 10% of the stores, and recorded conversations taking place in those stores.

Eh, even that is too strong. Discord already recorded the conversations and makes them publicly available to anyone who joins the server at any point in the future. Anyone who thought they were having an ephemeral conversation that wouldn't ever be seen again by anyone has never tried scrolling back on a Discord server.

right, which would be awful.

The researchers clarify that it's only those servers that are listed in the discovery tab - you don't need an invite link to join those.

> In this regard, this paper introduces the most extensive Discord dataset available to date, comprising 2,052,206,308 messages from 4,735,057 unique users across 3,167 servers – approximately 10% of the servers listed in Discord’s Discovery tab, a feature designed to highlight public servers that users can join.

Seems like this would be pretty trivial to reverse by simply searching for a somewhat unique message by a single anonymized user, you would then be able to quickly de-anonymize every user that was in a thread then you just graph that out across the entire network.

Reply on news.ycombinator.com