Profiling Hacker News users based on their comments
simonwillison.net
simonwillison.net
My first full time job (early 2000s) was working for a firm that did online cybersecurity related investigations for Fortune 500 companies (generally via a 3rd party law firm they had retained).
A big part of this was running investigations into people running "pump and dump" stock schemes on Yahoo message boards. We would generally start by scraping all of the posts for a user who had instigated one of these and then handing off the posts to an analyst.
It's amazing:
a. how much info people give out even when they think they are being careful
b. related to a, how even small tidbits combined over time can build a pretty accurate picture of who someone is.
e.g. they post "oh man, the Cubs lost", then a year later "went for a walk on Lakeshore drive", another year later, there was a fire at my local subway stop etc etc and you pretty quickly narrow down the rough neighborhood where they live in Chicago.
Combined with tools like Lexis Nexus and you get a list of people that you can narrow down by age, sex, occupation etc and we could narrow it down to <20 people based on other info they had shared.
Then you fold in their posting patterns and it's pretty obvious who is at work (posting 9 to 5pm) vs home (posting 7pm to 1am).
Again, you keep adding constraints and the intersection of the Venn diagrams gets smaller and smaller.
This was all in the early 2000s before we had cellphones that tracked your location and ad infrastructure that followed you around the internet.
What used to require a little work is now instant. And we're much further into the predictive part than most will admit.
Out of curiosity and without meaning it to sound like an accusation, did you write such similar posts by hand or do you use some form of automation for commenting?
It’s all manual and I guess just how my brain works. My wife actually calls it “the database” because I can quickly access stories and I apparently tell them in a very similar way.
I’m just as impressed that you noticed and had the Déjà vu.
In the sense that there were stories you heard retold (sometimes by the same person) over the years, mutating a bit in each retelling?
I think some brains get wired so that oral (or at least reproductive in some medium) story transmission is effortless, but affinity does seem to differ person-by-person.
It's funny you mention this b/c "yes". Both of my parents are big storytellers.
Never realized this so thanks for pointing it out!
I do this on Twitter.
Specifically, I'll retweet the thread or tweet I already have written on the topic.
Retweeting is, IMHO, the best part of twitter as it lets you "weave" a narrative of old tweets and threads. It's also why I think articles are dumb b/c you can't like to the specific part of an article like you can with a tweet thread.
> reply to tweets of other people with a relevant Twitter thread I had already written
I noticed I also have topics I talk about regularly and this would be really nice. Some things only need one high-effort explanation and then linking to that.
Using LLMs would in theory save some effort by rephrasing to it doesn't look copy pasted but I am strongly opposed to the mild reality distortion they are prone to doing (like hallucinating random tidbits which never happened, using "A big part of this" as a rhetorical device instead of an actual quantity, etc.) in addition to flat out lying and other mis-generation (I don't call it hallucinations since this is normal operation, not something exceptional).
I mentioned in a sibling thread that I do this on Twitter (and it's a lot of fun to get to re-use old threads for new audiences).
> Using LLMs would in theory save some effort by rephrasing to it doesn't look copy pasted but I am strongly opposed to the mild reality distortion they are prone to doing
Same thoughts from me. There is also a bit of "John Henry" [0] in that I want to keep my brain strong in this skill versus letting the machines take it away.
Being strongly private online requires spy tradecraft levels of precaution.
As for using clues to discover people's whereabouts and such: lots of police/detective shows have turned "finding where people are through Instagram photos" into a meme. Most people don't think about cybersecurity outside of "oh, I need to change my password now."
Curious about any recommendations from you or others.
> Most people don't think about cybersecurity outside of "oh, I need to change my password now."
Cyber anonymity is a concept that most people don't think about at all, especially post-Facebook normalization of real names ~10s.
In many ways, it feels like Eternal September talking to younger people who never used a pseudonym online.
Data might be a bit stale, but a helpful resource imho.
that's basically why clickstream analytics is so powerful
The first two, I've made peace with (nothing I can do about it anyway). The last one picks quite fiercely at old trauma that really makes me reconsider my socials in general, not just HN.
But maybe that's just the anxiety and trauma talking, encouraging me to recede back into the shadows and re-apply the old mask of "acceptableness" I've been trying to toss aside. Maybe the fact a free chatbot can do such a thorough analysis is in fact reason enough to stop worrying about every aspect of my identity and its perception by others, and instead just...be me, and deal with whatever consequences arise from that.
I dunno. Just...lot of emotions, here, most of them quite bad.
Anyone with access to a decent LLM can now perform a version of this in just a few seconds.
We desperately need to modernize laws around discrimination in light of the proliferation of these tools. No longer does someone need to thread the needle in interviews around "illegal" questions to find something to (metaphorically) hang an interviewee with, as these tools can pick it apart quite cleanly. People in protected classes are going to get reamed by bad actors leveraging these tools.
That said, after rubber ducking with a friend on this, I've come to the conclusion that there's two paths forward from this point: flight (scrubbing socials, hiding online, creating an acceptable persona) or fight (being firmly authentic, owning your weirdness, and accepting you can't control the outcomes of others' actions using these tools). I've spent decades in 'flight', and I'm tired of it. I can't control who uses these tools and to what end, so I may as well just be my damn self anyhow and do regular threat assessments accordingly. The more people who behave authentically, the less power these tools have over us.
The law, currently predicated on the difficulty of discriminating en masse without leaving a paper trail, will take a while to catch up with de facto use.
Hell, there's still no legal prohibition in most of the US on things like Equifax's salary history reporting. https://en.wikipedia.org/wiki/The_Work_Number
I taught infosec 101 course at a university ~20 years ago. (Twice.) On the topic of privacy I used an example of harvesting data on peoples' habits, movements and behaviours and then said that as a society we use two different terms for the same thing. "When an individual does this, it's called stalking. When a company does this, it's called data mining."
The economics department students, many of who already knew they would want to work in marketing, were quite offended.
But, the problem is real if it's a nation states or megacorps are doing it. They'll use such tech in an unjustified way, make a misjudgement, and then ask you to explain yourself out of the situation. Yeah, they're definitely going that, because they don't give a damn about it.
Because eventually apparatchiks with the data at their fingertips are going to use it to rule out the next Einsteins from participating in {insert major Chinese project}, and you've effectively self-selected at scale for people whose shared characteristic is "not being different."
We'll see if the US and Europe course correct on individual freedom enough to reap the benefits of that though.
Wow, I sound really annoying. Sorry about that everyone!
I knew the original!8-)) Hope you aren't him!
I feel the need to point out that 99% of the time that phrase is essentially an insult and isn’t indicative of a “nuanced position” lol it generally means “you’re myopic in your views/your argument lacks nuance.” That strikes me as a pretty charitable interpretation by the model there.
You seem like a good dude, and I’m not going to pretend I haven’t thrown out the flippant quip here and there in my comments. I just thought that interpretation was pretty funny.
> Simon Willison is a British software developer, blogger, and open-source advocate, best known for…
Holy moly - it was damn accurate. Those comments spanned the last few year, as I became less and less active in social media/online comments in general. But it picked up on me being a snob about unprocessed food, having hard red lines that I sometimes take a very harsh stand over, as well as so many other things.
I know I am quite open in what I write on the net and do not really fear future employers using this to qualify me - because if they do and deny me a job I know I would not have wanted to work there in the first place. And yes, this is a position of old white grumpy male privilege.
Still. It was impressive and "quite a bit dystopian". All in all, from posting the URL to Simon's blog post to being profiled took less than 6 minutes.
Create a new bookmark in your browser, name it something like "Profile HN User", and paste this as the URL:
javascript:void(function(){var u;var m=window.location.href.match(/news\.ycombinator\.com\/user\?id=([^&]+)/);if(m){u=m[1]}else{u=prompt(%27Enter HN username:%27)}if(!u)return;var msg=%27Profile this HN user: https://hn.algolia.com/api/v1/search_by_date?tags=comment,au...})()
If you're on a HN profile page (news.ycombinator.com/user?id=someone) it grabs the username automatically. Otherwise it prompts you to type one. It copies the profiling prompt to your clipboard and opens a new Claude conversation, just Cmd/Ctrl+V and hit Enter.
I created this account after using my real name here for years, to build at least some kind of separation. At the time, I think I was applying for jobs and had a couple of interviews - positive ones, oddly enough - where my political views were referenced. Given our political climate in the US, I decided it would be best to make at least my current views more difficult to associate with me.
For me, this just underscores the fact that while we always knew those data were out there for someone targeting you and determined - this makes it an order of magnitude easier to access.
… I just typed out an explanation why I made the above statement, but decided not to post it as it describes a potential criminal act that would likely be very profitable :(.
>The word "engineer" being diluted by software/bootcamp culture is something they return to obsessively — arguably their strongest ideological position alongside surveillance criticism
Busted!!
That being said, not surprised because it listed exactly what I want my persona to appear, does that mean I am like that irl? No, I rarely bring the above “engineer” term IRL let alone to be obsessed about it, but in HN it makes sense to bring up, rest are mostly about techie stuff that I usually don’t bring with my friends or family. Also, this can be about anything you produce, like your blog, books, YouTube, or anything, that personality is what attracts (or repels) other people to be around you, it’s human society 101.
“Your communication style is direct and often adversarial, using rhetorical questions and sharp analogies to pressure-test assumptions, with little tolerance for what you see as naïve, performative, or abstract reasoning. You prioritize competence, execution, and practical tradeoffs over signaling or theory, and while that makes your analysis grounded and often incisive, it can also make your stance appear combative and less receptive to edge cases or emerging paradigms that don’t yet fit established incentive structures”
I disagree because I tend to seek a middle way. I would agree that too much (excessive) introspection is bad. But I would argue that too little is equally bad.
I think obsessively examining ones own comment history would verge on excessive. I'm wondering how much LLM analysis of my public and private life can remain healthy.
If you write a journal, do you not sometimes review the things you've written?
This is no different.
If you read the whole journal every day before writing a new entry, then perhaps it is a but excessive. But a review once or twice a year is not a bad thing.
But to answer your specific question: I do keep journals and I almost never review them. I have a box with 20+ years of handwritten journals in it. Once every couple of years I'll even open the box and flip through a few pages of a few of the books and read a couple of pages. But that is generally all I do and I have no desire to do any more than that.
I also have maybe 100 journal entries in Google Docs and to the best of my recollection I have never read a single one of them after they were written.
Lately I've been using LLMs as a kind of active journal, where I engage in a dialogue as a means to externalizing my thoughts and refining ideas through critical feedback. I have never gone back and re-read any of those conversations.
I'm not making a value claim here, just answering you honestly. It isn't that I think one should or shouldn't review their journals, it is that it doesn't give me pleasure to do so and I find no personal value in it.
If you contact them and ask for your data to be deleted, they will directly refuse.
And while other sites mirroring HN might keep the original, that’s a separate issue.
I then followed it up with "Given my chat history, how do they compare to me?", and it started making comparisons of myself to myself. Very fun experience.
6. Slightly dry, understated style
The tone is usually compact, matter-of-fact, and occasionally wry rather than performative. Even their Wuhan taxi-driver remark has that deadpan “I have seen the circus” flavor.
" The Atari + German book reference indicates:
* Interest in legacy computing / systems history "
No. I'm just that old. I read the book when the Atari ST was state of the art :-)
https://antirez.com/hnstyle?username=pg&threshold=20&action=...
Which lets you find the alts of a handle
> Old man raising fist at, and yelling at, clouds. Get off his lawn.
[^1]: not really - this is speculation (so... kinda the same thing the LLM is doing) but is possibly an accurate representation.
Edit: turns out it's case sensitive.
Sounds about right:
roughly “anti-imperialist realist” with Indian/Global South anchoring and paleoconservative/libertarian-adjacent distrust of state-corporate surveillance power.
They will not delete all your comments. They might agree to delete a very small number of comments.
That does not help much in the profile-building/AI perspective.
They should be transparent about this upfront, on the signup page, but I suppose that would hurt conversation stats.
Cliché but true: On HN we are the product.
Edit: my use case is building a graph for archiving every link ever posted on HN (posts and comments), if that’s relevant. The contents of HN comments have little value to me for my workflow, nor do I profile users.
Surveillance was always possible but it was expensive because a person or two had to literally watch a suspect. So it was rare in practice and suspects had to be chosen carefully.
Mass surveillance is not new in the sense that surveillance was always possible. What is new is the scope of power it gives to those who can use it.