Mapping Hacker News to find who knows what in the HN community | Hacker News Reader

Mapping Hacker News to find who knows what in the HN community

blog.wilsonl.in

379 pointsrobg212 comments

Personally, I like how HN focuses on content and discussions rather than individual users. If I wanted to follow experts, I'd probably curate a selection on a social network like Mastodon, or kludge together some RSS feeds.

Also, I feel like this tool selects for active commenters, not for knowledgeable experts. Not to mention throwaway accounts.

Still a cool project.

Thanks! Those are fair points. We're thinking we could uplevel the social layer so you can connect with people of similar interests for deeper connections. In this way we compute not just your contributions but how they relate to others.

With such efforts I think it is time for people start deleting their accounts, not because they want/need to hide anything, but some people may want to stay semi-anonymous by using aliases and data mining everything and correlating it with other sources (e.g. LinkedIn) may help identify them and cause trouble (e.g. someone wrote something about their workplace without naming it, but hey, it is "John Doe that works in MegaCorp".

> it is time for people start deleting their accounts

HN does not let you do that though. At least last time I asked about it, they sent me a response saying that they’d notify me when it became possible to delete an account. And they provided some reasons for why they don’t delete comments and accounts.

It would destroy older comment threads. There are quite some gold nuggets to be found there.

Sure, but what is more important? Some random comment threads, or honoring the wishes of the people who have been contributing comments to the site?

For me I would prioritize the wishes of the contributors.

Well, how about the contributers honoring the wishes of the owners of the plattform?

Also my wish as a contributer is that the threads stay like this. So I can come back later and reread them. I often gained value in reading a linked old thread.

My advice for people not wishing for that, would be simply to stop commenting instead of demanding the site should change.

Could there be a middle ground to remove the account ID as a correlation between posts? The posts could remain as a graph of replies without retaining who posted which.

I know this means someone could still use stylometry to try to reaggregate the posts, but that's less reliable than HN actually telling us that the same authenticated user made all these posts over a decade.

I could very much live with that solution.

I thought it goes without saying, that when you post something sensitive, that you don't want to connect openly to your account - you use a throwaway account. HN made it easy to do that.

"Witness Mr. Henry Bemis, a charter member in the fraternity of dreamers. A bookish little man whose passion is the printed page, but who is conspired against by a bank president and a wife and a world full of tongue-cluckers and the unrelenting hands of a clock. But in just a moment, Mr. Bemis will enter a world without bank presidents or wives or clocks or anything else. He'll have a world all to himself... without anyone."

-Twilight Zone Episode: Time Enough at Last-

The social web died, all you're doing it making pitchfork and torch 2.0 for mobs.

If you want to add value and not bloody public spectacle rank comments instead of users.

I have a bunch of low quality posts here when idiots piss me off, but also share world first research and breakthroughs I've been involved with the rare time the counter party is worth talking to.

> rank comments instead of users

> I have a bunch of low quality posts here when idiots piss me off

It’s ironic because in the second of the parts I quoted you on here you are basically yourself generalizing the users instead of the comments.

Is it really “idiots” that piss you off (the users)? Or is it the specific things they said in isolated cases (the comments) that piss you off? Wasn’t your point exactly that this kind of distinction is important?

And the OP here seems to willfully ignore the main point his parent was making. He said that not having this focus on a "social layer" made HN better. But then OP says:

    We're thinking we could uplevel the social layer

head => palm

It’s great that you finished it but I wish you had taken a little more care to protect users here. Some people I respect a lot are a little more vulnerable because of this

How?

This seems like a slippery slope towards bog-standard social media.

Currently, HN is the only place on the internet I am willing to interact with others _because_ it lacks the "social layer" you are recommending.

The focus on user comments that are thoughtful, relevant, and respectful _is_ the social connection I value.

I concur on the sentiment - I think it’s great we don’t have avatars etc here as it makes it more content focused -

But at the same time I think the OP has made an awesome project! Well done

I agree. It's also quite humbling which I think is a good thing.

I think that especially in CS, since applications of which touch on nearly every possible field of knowledge, computer scientists often run into trouble of assuming they know more than we do.

CS people are prone to the engineering trap of "I've learned one slightly complex thing so obviously I'm capable of knowing every complex thing". It tends to forget how important sheer quantity of practice is in every field that expects a higher than high school level of education.

The negative space should be the most interesting, since absence trends are the hardest to recognize. What are Hacker News’s quantifiable blind spots? Answers on a postcard.

I've been interested in a map of what gets flagged for a while.

Not the negative space, but a list of untolerated content.

Personally I don't read user names. Easiest way to focus on the comments. Of course, the lack of avatars and signatures and stuff like that helps a lot.

I have absolutely no idea who I've had an argument with on HN ever. I'm sure I've had a few.

It's interesting to keep track of some known people. Sometimes you get to see (for example) a thread with cperciva, tptacek and animats - and I think it makes it more fun to read when you know...

But but ... they're known to you because you read the user names ...

I’ll often looks at the username if I find a comment either particularly insightful, or particularly stupid. It tends to be the same small collection of users, so I guess I’m agreeing/disagreeing with people quite consistently.

There is one person here that is like my Polish doppelganger. I'll go to post in a thread and I'll find they already posted the gist of what I was going to say.

There's also https://hnrss.github.io where you can "subscribe" to a user's comments.

If I see a thread with too many comments from high volume posters, I assume it's low signal. Most of them are very good at commenting in a style that is convincing sounding and popular here, but when they stray into topics that I know well, I often find them to be confident, plausible, and wrong. (And also upvoted above people with correct information who don't have the same name recognition.)

>I often find them to be confident, plausible, and wrong. (And also upvoted above people with correct information who don't have the same name recognition.)

This is basically 99% of hacker news comments in a nutshell.

You pass the confidence, plausibility, and accuracy challenge with flying colours!

I'm sure that happens with me (I've noted for awhile that banal comments I write here get upvoted, presumably from a combination name recognition and people who engage with HN by following specific people --- which is how I do it too). What I'll say is if you see this happening with me, call me out on it! Refer back to this comment, if you like, when you do.

Apart from sheer volume (I have been a message board nerd since FIDONet when I was a teenager, it's second nature for me) I attribute what success I've had here to engaging principally on security topics, which is where I've spent my whole career. But I also just like to shoot the shit about stuff! I'm wrong all the time! (Including about security stuff).

One username that always stands out is “1970-01-01”. Every time I do a double-take, thinking a timestamp has gone wrong.

the same, I get weirded out sometimes when I read a remark from someone who evidently has decided to keep track of who I am on the site. Seems like a lot of wasted mental effort for little reward.

A couple of weeks ago someone made a whole anonymous Mastodon account just to ask me a vague question about a comment I made here on HN like six years ago. It kinda creeped me out.

A fan? A stalker? Just a rando with too much time on their hands?

I do look at the commenter's name (same as for an article's author) as I know quite a few commenters personally (some are former colleagues) so our replies sometimes refer specifically to something the other person knows or did.

Obviously this applies to a negligible percentage of total commenters, but as I only comment on certain topics I’m more likely to encounter friends with the same interests/experiences.

But curating such a list of experts to follow takes quite some effort. It would be great to have a tool that helps with that.

And sure, ideally you wouldn't need such a list of trusted experts but just focus on content. There even was a time when this worked - you could just type "what is the best database to use" into a search engine, and get a helpful result. Not anymore. On HN it may still be better than elsewhere, but ultimately it's a similar issue.

https://hn2.wilsonl.in/user/simonw includes "Risk of COVID from pianos" down at the bottom of the map. I'd love to know where that came from!

Appears to be a summary of this comment: https://news.ycombinator.com/item?id=30105274

If you zoom and pan the red cursor over that map location/text, your source comment will appear underneath the map.

Great call out! Would it help to make the points more interactive?

Can you please remove the email address from the user profile page?

I wonder if there's a risk of pianos from COVID?

Kind of like how when the pandemic started, sales of guitars and home gym equipment went up as people worked on at-home hobbies.

Mine seems fairly accurate - accelerating RDP, 3d printer firmware, touchscreen Graffiti gestures (which kind of dates me) and mainline Linux in SBCs, all of which are my hobbies (I seldom, if ever, comment on actual work, like cloud architecture, networking or... AI, which is too crowded a field).

Weirdly, almost no mentions of Apple stuff (which is supremely ironic given my blog).

So you get an impression of what people discuss here, but not necessarily what they know :)

https://hn2.wilsonl.in/user/c0l0 - Apparently, I'm a (the?) leading expert on Optimizing Toilet Lid Design - AMA.

Puh, that's nothing. I'm world reknowned for my Fireplace Efficiency Comparison analytics.

(I wonder if the embedding was generated using IDF, placing maybe too much emphasis on rare word tuples?)

Mine is an absolute mess of nonsense. I suppose it makes sense, so am I.

https://hn2.wilsonl.in/user/SketchySeaBeast

"Dual Flush Toilet Button Design" authority here, so back the fuck up, I guess

How are you determining what you’re specifically known for?

Mine has a ton of stuff:

https://hn2.wilsonl.in/user/AndrewKemendo

Recent and related:

Show HN: Exploring HN by mapping and analyzing 40M posts and comments for fun - https://news.ycombinator.com/item?id=40307519 - May 2024 (159 comments)

Thanks dang! Amazing to me how much HN brought Wilson and I together after 16 years and across the world. We're hopeful about the future of trust on the internet exactly because of communities like HN.

So, you I assume you extracted my mail out of the profile text (which admittedly was not obscured enough for the LLMs of today), to put it into a mailto: link - Well, many thanks on behalf of the low-effort spammers for making harvesting easier, I guess...

lol, it failed at decoding base64 in my case

I do wonder if low effort spam bots scraping mailto links are still a thing or if we are all just cargo culting obfuscations of emails. Spam has changed so much from those days, is it a strategy that is still used?

I usually don't mind these things, but that's not OK, did the same to mine.

Is there an opt-out of this thing?

I think even basic LLMs are able to de-obfuscate our 2000s-style attempts at hiding emails (eg. one of my friends writing: "firstname [cute arobase sign] domain.com" on the contact page of his super-popular blog)

I remember there was a (rather controversial) tool posted here a few years ago, which used textual analysis and stylometry to find “similar users.” So you could type in someone’s username and find their likely alt accounts. It was creepily accurate - or at least that’s what I heard from a friend who has an alt :)

Could this tool be repurposed for that? Presumably the “map” rendered in each user’s avatar could be encoded as a vector and then compared to that of another user.

EDIT: Wait, I just realized it already does this… (or at least I think so - it’s not immediately obvious if “Explore More Users” is ranked by similarity.)

Interesting! Seems creepy to me too, as you can find it would love to understand how we can do better.

> Could this tool be repurposed for that? Presumably the “map” rendered in each user’s avatar could be encoded as a vector and then compared to that of another user.

It's probably less likely to work because people often use alts to participate in discussions that they wouldn't want to associate the primary identity with. Whether that be discussions about their employer, about politics, or something else, the subjects an alt particulates in will likely be different.

Style works because few people are capable of fundamentally altering their style even if they tried.

My final year project in uni was a tool to detect plagiarism by analyzing whether parts of an assignment had been outsourced to anyone else (basically did one author write the document on their own). Or if two projects by the same student were actually written by one person. For what was at the time an extremely naive implementation of various stylometric methods it worked surprisingly well. Even I was shocked when I managed to demonstrate the tool working. People should read about this branch of studies if they care at all about anonymity online.

Any interesting links you can share for starters?

Lemme see what I can dig up from the past. I’ll mail it you if I find anything

I would be amused to see whether or not you could simply mask all of your comments by filtering them through a commodity LLM or not. I would assume so.

mixtral-8b: "It would be entertaining to find out if you could conceal all of your remarks by running them through a run-of-the-mill LLM. I suspect that you could."

nemotron-4-340b: "One might find it intriguing to observe if you have the capability to disguise your statements by utilizing a common LLM as a filter. It's plausible to believe that you could."

gemma-2-27b got verbose on us: Here are a few ways to rewrite the sentence "I would be amused to see whether or not you could effectively disguise your comments by utilizing a common LLM" to reflect different styles:

*More formal:*

* "It is hypothesized that a commodity LLM could be employed to effectively obfuscate your comments." * "Utilizing a commodity LLM for comment anonymization is a plausible strategy."

*More casual:*

* "I wonder if you could fool a regular LLM with your comments?" * "Could a basic LLM make your comments look like they came from someone else?"

*More skeptical:*

* "While it's possible, it's unlikely a commodity LLM could effectively mask your comments to the point of complete anonymity." * "Are you sure a simple LLM can truly hide your identity? It might not be enough to protect you."

*More direct:*

* "Using a commodity LLM for comment anonymization is a viable option." * "Can commodity LLMs effectively anonymize your comments? Probably."

The best way to rewrite the sentence depends on the context and the tone you want to achieve.

----

All of the above courtesy of perplexity.ai's playground.

What a fun test!

I believe this was it: https://news.ycombinator.com/item?id=33755016

Sad, the website is not more.

The OP got what they needed out of it after people arrived at the site and searched for multiple usernames in quick succession :)

Ah damn.

I built a system that did this years ago https://x.com/austingwalters/status/1041894765439201281

Ironically, it doesn’t use embeddings even though at the time and since I’ve basically exclusively used embeddings.

Main issue with embeddings is they don’t change well overtime (you can adjust, but not as easily as other methods). Language is filled with industry specific acronyms and models are generally not great at adapting to changing phrasing and acronyms.

Anyway, eventually decided to put some more pieces together and built it into a company - https://ipcopilot.ai

Now we discover ideas on hacker news for our demo https://m.youtube.com/watch?v=B5ymgh-ZDiI&pp=ygUKSXAgY29waWx...

I think it would be interesting to, sort of, summarize users.

Like user123 is moderately funny, articulate, interested in typescript and web development, and rated high on conscientiousness and extraversion.

It'd be interesting to see how the LLM sees us, vs how we see ourselves, vs how other HN folks see us.

I chose my username from the narrator's alter ego in Zen and the Art of Motorcycle Maintenance. In reference to the analytical knife:

"Phædrus was a master with this knife, and used it with dexterity and a sense of power. With a single stroke of analytic thought he split the whole world into parts of his own choosing, split the parts and split the fragments of the parts, finer and finer and finer until he had reduced it to what he wanted it to be. Even the special use of the terms "classic" and "romantic" are examples of his knifemanship."

In a bit of nominative determinism, or perhaps just having chosen the name because I know myself (or maybe I just over use these words), my keywords include: "part, system, level, language, article, object," etc.

I found it a bit challenging to actually drill down into my own username, but it doesn't seem to offer much other than throwing a lot of dots all over the map. I'm trying to understand what the overall clusters might be, but most of them are just android/apple/google?

https://hn2.wilsonl.in/user/mmastrac

So my expertise in apple, google and tesla is not so unique?

At least you have some expertise... Mine comes up blank...

Interesting. I have noticed that my most-upvoted comments relate to legal questions, which I have relatively more expertise than most HNers (I used to be a lawyer). Although I'm not among the top commenters for law/legal/lawyer according to this tool, I definitely recognize some of the top names and can recall seeing their comments in legal threads. Pretty cool tool!

Does the HN API include comment score?

No.

Wait, then this tool is based on volume of references?!

That would be weird. I rant a lot about spirituality and religion and it doesn't show up. But that one time I talk about pregnant man emoji and now I'm the reference for that.

Very interesting, thanks!

The best thing about HN is that comments feel pretty temporary. I don't like the fact that I'm being analysed without my consent and put on public display. Not that there's anything remotely interesting about me on that page but it feels weird. Not everything needs to be analysed and we don't need to compete everywhere.

What I'm saying is: I like the focus on the content and that it's not about who said it.

It got me to remove my twitter handle from my bio though. If you could update that in your app I would be thankful.

> The best thing about HN is that comments feel pretty temporary

I have a google chrome extension that lights up when there are HN comments for the page I'm on. If I organically discover a blog post and that lights up, then I will often read the comments even though it might have been posted years ago. There is often useful insight or context to be gained.

FYI to you and anyone reading this. Comments are the opposite of temporary. You can never delete them or edit them after a very brief initial window. So anything you say here can and might be associated with you forever. Either directly or nebulously as indicated by this tool.

I suppose the EU would like to have a word about being forgotten?

Just because an individual has the right, doesn't mean the other entities have the means or desire.

Pithy!

But I guess I'm still confused. If a vendor provides software that doesn't honor the right, how many times can they do that before they get in trouble?

In HN case, I'm sure an email would sort it out, however.

If they're not in the EU, probably "forever," especially if they are in one of the many countries that are politically indifferent or hostile to the EU.

Information flows freely worldwide. Regulations do not.

Since HN does not ask for a name or email, it's difficult to say that one is being 'remembered' by their comments.

If your username is your full legal name and city, I would say that's your own stupid fault (and I live in a GDPR country).

This makes sense. Thanks!

Yup, thanks for the reminder! I actually knew. And I'm sure they don't serve GDPR requests. But the comments become irrelevant after 24h so it feels like they aren't really temporary. It's always about the newest articles and stuff

> ...comments feel pretty temporary.

I know how you feel.

I felt that way, too.

But what I found was that 2 clicks away from any comment is a query like: https://news.ycombinator.com/threads?id=RamblingCTO

Comments here have never been temporary.

Hacker News itself has for many years provided a free near-realtime API and a variety of large datasets to the world for the specific purpose of making it easy for anyone to analyze our comments here. (https://github.com/HackerNews/API)

No comments you write on the Internet have ever been temporary. They stopped even pretending to be so 20+ years ago.

Yes I know that. But they become irrelevant pretty quick. It's not like someone comments on a post from a year ago typically. That's what I mean

Interesting, but

  Rather than organizing the world's information, what if we could organize the world's people?

I do not wish to be organized.

As a not so active user, this tool is rather inaccurate. It seems to have focussed on the one question I asked about jpeg xl, which is the topic I know the least about.

I suspect a bias towards more common topics might be occurring.

Things I thought I had expertise in and have posted about disproportionately on HN: chess, Python, HFT, Fermi estimation, Ireland.

Keywords the site actually associates with me: language, English, article, team, book. To be fair, at some zoom levels I do get "chess".

What's up with dangs profile?

https://hn2.wilsonl.in/user/dang

Why does it say "Marion Milner" and why are there only so few posts on the map?

Ha, I had the same thoughts! We’re working to do better, would love your feedback.

It’s the only name in his bio. He quoted Milner.

"Tax avoidance schemes in The Netherlands"

https://hn2.wilsonl.in/user/dep_b

I guess that's why I still like posting anonymously?

This implementation doesn’t really get there, but…

I think this is an extremely cool idea both on HN specifically and generally on the Internet. Bluesky does a bit of the thing where you can mix and match your content to your ranker/recommender system.

I hope you folks keep working on it, this is a refreshingly cool hack in the space.

Super cool! Happy to see that I show up under documentation-related stuff, considering how frequently I mention that I'm a technical writer in my comments. Also pretty excited to connect with the other people that show up related to docs / technical writing / etc.

As opposed to me not showing up at all for things I comment on. Guess I'm not knowledgeable enough.

The results are biased towards users with high post count. It doesn’t matter how knowledgeable you are about a topic, only how much you comment on it.

And using the correct keywords. SEO is dead, long live SEO!

Also I imagine that my area is pretty niche (not many people talking about documentation on HN) so I may have less competition

We'll take a look!

> "Despite the intervening 16 years, we're amazed that social networks, even Hacker News, don't compute and display the trusted voices across topics. Instead of prioritizing pages based on content, social networks could prioritize the people behind the content."

Fallacy: appeal to authority. Practically, just because someone generates great content on subject A doesn't mean their take on subject B is any better than random. A well-reasoned self-consistent argument informed by accurate data is far more valuble than 'trust this expert opinion because this expert can be trusted' approaches - although it may require more work on the part of the reader. Don't get lazy.

We're trying to stay away from authority and just compute based on what you say then transparently show where that information came from. How do you think we can do better?

Some sentiment analysis might help? Apple is one of my keywords, but you'd pretty-much only get an "avoid" response from me (to a degree, I have recommended their phones to a very specific demographic).

Awesome project, fantastic UI.

Same. I think a sentiment analysis could be helpfull.

I find myself dragged to Apple discusions more than I would like to admit, but they are generally negative in the last years.

It's a topic we keep coming back to, seems like we can do better than sentiment analysis to show the semantic space of the sentiment(s) and how those breakdown by users and groups. For instance assuming a large discussion set of apples, different types of apples, and other types of fruits, we'd discover the patterns among users. Have you seen folks doing that type of semantic sentiment analytics?

As Wilson and I are working on this project, we'd love to hear your thoughts!

The visualization of the space is excellent.

Thank you!

It's very fun to play around with, there seems to be value somewhere.

I wonder if you could make a video of how an individual could use it effectively in a pragmatic sense e.g. networking, research

(after more playing around I get the UX, very powerful indeed)

(feedback: add some controls for filtering (dates etc), relevancy, color islands slightly, just general UX improvements and I would probably pay a few dollars a month for this)

(feedback: when looking at a user, you have to zoom really far in before results start appearing. it should probably show labels/results if less than X amount of data points are in your viewport.)

(feedback: Is there "Explore more users" section the more related users to the current profile? It would seem so, but not immediately clear. And if it is, none of my related users have me as a related user in their profile.

===

This is really inspiring work, hats off!

Thanks so much and for the great feedback!

Thanks for making and sharing!

Could you share some info about how you generated the 2D space embedding/visualization?

Thanks! I'll ask Wilson to say more in our next update.

OK, I'm going to just be upfront here and admit that I'm an idiot. This is the third time I've seen this link come up on HN, and each time I've checked it out, but I can't really figure out how to interpret what I'm seeing there in a way that is useful.

How can that map be used to determine who is knowledgeable about what? Looking up myself, I can't connect what I see with my own areas of knowledge.

I think I need an ELI5 for this. Again, this isn't a criticism of the effort at all, it's a public admission of my own ignorance.

Take everything a user says and encode it into some vector. That's "embedding."

Then you encode a search query the same way and return the nearest N vectors/users. That's vector search.

It seems they did the above for HN comments.

Right, I understand all that. I just don't understand how to interpret the resulting map (or how to formulate a search) to provide useful information, specifically "who knows what".

Thank you for being so clear on where we are failing you. We're basically computing your semantic space as compared to the whole community. The map shows the relevant instances in that space. As you browse your concepts the space changes and we link to the relevant comments from the associated threads. Who knows what is prioritizing the user and the semantics / voice of that user amidst the noise of the whole community speaking at once.

We'd love to hear your feedback as you play with it more and we improve all of the above.

It obviously isnt intuitive, but I suppose one could learn to read it over time. For example, there is a lobe at (5, 10) that lights up like a Christmas tree if a user was engaged with Covid vaccines and mandate discussion. Of course, it seems like there is no discrimination between the different stances a user took, or the veracity of their commentary.

> How can that map be used to determine who is knowledgeable about what?

To be fair.. you can’t. Unless you think that mentioning specific words or phrases associated with a specific topic a high number of times means that you’re knowledgeable at it.

Looking at my own profile the words (topics i know about?) seem sort of generic. "issue", "company" - not sure what to make of that other than I assume those are words I use a lot in comments?

We'll take a look at how to do better, thanks!

I feel targeted when you call me a "system server user network number service code issue" ... not that it's inaccurate ;)

Ha, exactly!

Is the knowledge graph weighted more to submissions than comments? I'm surprised to see how generalized the keywords associated with my user are "Company, Product, Market, Part, Article, Issue, Bit".

I'll admit, I am a generalist, and I comment more than I post, but I would have thought/hoped my knowledge base would trend towards neurotech, sleep, mental health, health, wellness, etc, as I feel that is the "less general" stuff I comment on.

Though I recognize this is how I like to see myself, I do often comment on business models, branding, etc etc, which are areas I have interest in.

I wonder if topics that have more activity from a user, but are in a less popular topic area would improve this?

Right now it does end up representing quantity + quality than just quantity alone. How can we do better?

How are you measuring quality?

I'm thinking you'd have a huge volume of people with similar common attributes, and if these most common attributes/keywords suggest a more baseline understanding of the community as a whole.

I'm not sure how you surface the more nuanced understanding of each users knowledge or interests.

The few things I'm thinking are 1) more weight to more recent content than earlier. We change and grow as people. I used to be in the music industry, then 3d mapping, and now neurotech. My music experience is now old, but neuro is fresh. That may be more interesting/valuable.

2) commonality with the viewing user. Where do we overlap, and again, where do we overlap where others don't? How often would a person be searching for someone whom they don't have overlap with? We are likely interested in the same things. Though perhaps somebody who knows nothing about marketing, is looking for marketing help, so I'm not sure how you surface that.

I guess I'm wondering what you imagine the ultimate use case being? Or the early adopter use case. You may be prioritizing the algorithm for uses I'm not expecting.

When you click on some keyword it's actually to go back from that keyword.

It would be nice if map showed titles of all articles on extreme close up.

Overall it's interesting tool but I think for people with small amount of comments this is not much informative.

How does determine which comments are valuable ("trusted") without access to the vote count of comments?

We're trying to stay away from karma and focus on what people are saying within the community, rank orders, overall semantics, and more. How do you think we can do better?

I'm ignorant on how you are measuring quality, but I think an interesting blend of karma with whether or not there are replies that argue. If the replies are generally positive, that may indicate quality. If the karma is below 1, and the replies include a number of refutations, that can indicate low quality.

I don’t think comment “score” is available in the api.

Correct it's not (at least that I can tell), but there are some (indirect) ways to approximate that number based on order, time, and/or color

Yeah I was thinking this as well, though the indications of color (maybe dead-ness also) would probably have to be scraped from the front-end.

Would be interested in a study, unfeasible as it may be, on the social compatibility of people who have the strongest overlap between themselves and the next closest user.

Cool visualization and analysis, really well made!

Thanks so much! How do you see that study playing out in metrics that we can measure and examine?

Email adjacent users with a chat room link, say what this is about (you were identified as being similar, maybe you have similar interests)

Email random users (control group) the same

Ask after a month who's still in touch

I'd participate but please send me some of each group and don't tell me which is which :D

Absolutely no idea :D but if you were able to quantify it somehow I think the results would be fascinating.

What instructions did you give it for the bio? It tries to decode a rot13 email address (and gets to a recognisable point) but then says "no bio provided". I can't figure out what it's trying to do

Also, thanks a lot for attempting to put it in decoded form on another website. If I now get spam on that address, at least I know whose idea that was and that it's not yet spammers who got this clever, but rather it's due to well-meaning hackers with an idea: the most dangerous kind! :P

When I did this, I found it to also be an interesting way to fingerprint users and find alternate accounts. I was able to match an old account I used in a top 10 similarity match out of all users.

This is really cool. Also a healthy reminder that anything we post publicly is likely to be analyzed. With a little analysis it's probably easy to know me better than I know myself.

Maybe also who you'd tend to get along with and your unique place in the community?

while I can appreciate a fair amount of work went into this, it's not providing any useful information on users, or HN generally, that you couldn't more easily reach by searching by username on hn.algolia.com

Somehow I'm #10 on startups. https://hn2.wilsonl.in/search/Startups

I guess having over a decade in startups counts for something, but it's crazy to score higher than sama or all the other founder/investors here who make 1000000x more from startups.

My favourite terms here are apparently "inclusive pregnancy emoji proposal", "Controversial tweet on China issue", and "enhancing pasta flavor with salt". I do remember all of these conversations, it's interesting to see it brought up.

Anyway, I'm gonna bookmark this for the next time I look for a job on HN lol

It’s funny how the market works that way, right? The smartest engineers I know aren’t the ones making the most, and that carries for a lot of other things too.

Though if someone were an expert in business or startups or sales, I'd say income from it is a good measurement stick.

Looked at mine. Says not much at all about “what I know”.

Checked RIP Aaron Swartz. Nothing meaningful. What am I missing?

https://hn2.wilsonl.in/user/aaronsw

We'll look into this, thanks!

I tried a few topics, e.g. neuroscience and language (a minor variation of the query in the blog), and the results are rather useless. It certainly doesn't show users with knowledge of the topic. Perhaps too fringe?

I tried three searches'autism' 'episodic memory' and 'hippocampus' and I was pleased to see my account on each. I guess I have my niche :)

If you select some of the keywords (e.g. "brain" and "linguistic") for a user, you get posts they made related to those keywords. I think the results are okay in terms of relevance to the query, but of course "posting about a topic" and "being knowledgeable about a topic" are different things.

Now if this tool could distinguish posts that sound like they were written by a layman from those made by an expert...

Just curious what the mechanism is for adding a bio to individual users. Your sample -- robg -- has the same bio on your stuff as on HN but others have stuff on HN and "No bio provided" on your thing.

Right now if you add it to your HN profile we'll be able to include it.

One data point:

I checked my profile earlier and there was none, though my HN profile has scads of info. That's why I was curious.

Totally fair to say "Word limit!" or something, but it means your reply doesn't seem to jibe with my observation.

Thanks, we’ll take a look!

Seeing my domain knowledge mapped out so honestly, I am reminded of the line in Robert Burns's poem "To a Louse":

Oh, would some Power the gift give us / To see ourselves as others see us!

For the “terrain contours” is there something specific you’ve done to make it feel more cartographic? Or is it basically just marching cubes / iso lines on some data points?

Looks fantastic. Very cool project.

I'll ask Wilson to chime in on the question; thanks so much!

I hate this. If i used my real name id have some pleb website on the internet making chump guesses at what i know based on what i was prepared to share. Thanks for reminding me to use anonymous burner accounts on everything app i use.

Thanks for the feedback. How can we better support authenticity and trusted voices, where you fit in, and others like you?

Dont bother. Im never going to be your demographic. I don't want to be profiled by anything for any purpose especially when 1. that profile is then shared publicly 2. I never got consent. I don’t need tailored recommendations and im not going to assess people based on a third party’s black box interpretation that cant possibly have appropriate context.

I'm not quite as extreme as OC, but value privacy enough to not carry a cell phone and always pay cash; my suggestion would be to allow an ability to opt-out, which would unfortunately probably require you to retrain your entire model (i.e. to destroy all connections/weighting to the opt-outs).

An example would be that users place a small opt-out snipped within their user profile, e.g. [no-robots] ?

Perhaps OC might consider never commenting outside of 4ch..? Except even glowies train LLMs =D

I share on hacker news because:

1. I want the knowledge/experience/anecdotes/opinions of others. 2. I want to share my own knowledge/experience/anecdotes/opinions with others. 3. Contributing low-bandwidth, high-quality* content is (hopefully) doing my small part in encouraging others to do the same.

My primary concern: You are providing a map of individual user interests over a period of time. This is feels like a violation of privacy to some and is outright dangerous to others.

Why? 1. People have many interests. The Marketing Manager likes birdwatching, Bollywood, hacking podcasts, amateur chemistry, traditional Ukrainian breadmaking, ASMR, erotic independent film, rock climbing, pole-dancing, gunsmithing, and horse racing. 2. People are public with some interests, and private with others. The degree of public vs. private disclosure can come from external pressure (ie: political oppression, public servant expectations) or internal motivation (ie: embarrassment, pride). 3. People change over time. The 27 year old Marketing Manager may want to distance themselves from their poor political comments as a student.

Where to now? If you stop work now, someone else will pick up where you left off. So how do make a system which balances the user AND allows you to continue?

1. Easy and simple opt-out option. 2. User profiles. Allow me to share /which/ interests are associated with me. I am a nuclear engineer by trade, but I'll be damned if I want to talk about it in my spare time. Let me opt-in to Birdwatching, and opt-out of nuclear engineering so those weirdos at work don't find me. 3. Limit the Time for association. Maybe that means you only 'connect' people with their comments in the past year as the Public default. Once the user connects with your system and has control over their online persona, then the user can decide what to show and for how long.

Future use? 1. Push this tool as an anonymity improvement tool, which also encourages meaningful conversations with people you want to connect with. 2. Feature: "We notice you posted for the first time to /r/bsdm with your /u/PepsiOfficialMarketing. Was this a mistake?" 3. Feature: "Our AI has scanned your recent posts. We have connected your writing style across two accounts and this may compromise the seperation of accounts. The following phrase: 'Do what thou does MFK!' was used to connect these accounts by our AI."

(*Compared to current internet gasoline fire of ad-infested, algorithm targeted, Top-10 lists of Top Ten Lists.)

So cool. Love seeing this novel pivot of HN data. Just confirmed that my comments are well embedded by testing a few topics (longevity and mouse biology). Privacy issues not a concern for most academic researchers (we typically beg for PR), but I can see that this might contribute to more cautious wording.

I left Twitter right before it became X, and focused on HN as a “breaking news and commentary” that is mercifully free of politics.

This tool fills a discovery gap.

Thanks so much, this is exactly how we feel!

When i looked up my name, the word "argument" shows up...i was kind of hoping i knew more than just how to argue ;)

In any case, interesting idea & project. Philosophically i'm not sure i like the idea of identifying experts - i'd much rather people's comments stand on their own instead of their clout, but nonetheless definitely interesting.

> Mapping Hacker News to find who knows what in the HN community

Would things like this create/increase incentives to game the metrics?

In my limited experience, people tend to gamify any numeric metric readily apparent to them.

Anecdotally, total game time (in hours, usually) is used to convey experience in video games (WoW, CS:GO, PUBG). I've seen people create & run 3rd-party software to artificially inflate these types of metrics.

And some metrics aren't just a "game", but are gamed for very real benefit (e.g., gaining admission to a college, getting a promotion, being more employable) in ways that are counterproductive to pretty much all other goals.

So, when something like HN has goals, and you create, say, something akin to a contest and link it to real-world benefits (e.g., people expect a recruiter or hiring to use that "who knows what" info), then many people will modify their behavior, probably to the detriment of original goals.

We're trying to stay away from karma for that reason and focus on what people are saying within the community to find trusted voices. How can we do better?

How can Web sites do better at producing valuable and trustworthy content, when what everyone cares about now comes down to SEO?

Why? All my game times are inflated because I either don't shut it down when i go away or I alt tab out of it, do something productive and forget about it :)

Is it who knows what? Or is it what's pushing whose buttons?

I looked myself up and "google" is proeminent. However I'm only posting anti google comments, nothing technical but on the privacy theme...

I also looked up 'yocto', which is something i know something about and i mentioned in posts a couple times, and the first user returned has some very interesting tags:

gur juvpu ohg guvax evtug qbrf

And the only really related tag i see is 'buildroot'. I guess it's just not a popular enough tag for the machine to have enough data.

Edit: and to join the choir of concerned voices:

1. It's not 'trusted'. It's at best 'popular'.

2. It reminds me of the main social networks and 'engagement'. Hope HN never becomes as predatory.

>gur juvpu ohg guvax evtug qbrf

ROT13 for "the which but think right does". Not really any less cryptic even when decoded.

I guess mundane words in a ROT13 conversation people had once have attracted the keyword detector.

I thought of rot13 but then i read them as separate keywords and just assumed a glitch in the "AI".

I suppose yocto does sound a bit like rot13...

Searched for bingo and didn't see patio11 in the results; hmm...

It puts me in the top three for "theorem proving", and at the top for "interactive theorem proving".

I like it. I feel this tool is quite sophisticated and incredibly accurate.

In your avatar, what do the size and positions of the circles represent? It's not super obvious to me how that correlates to the word cloud on the map (if it does).

https://hn2.wilsonl.in/user/rob

Seems pretty accurate. I do love me some PHP and WordPress.

Good to hear!

Also, love the username. My last initial is G as well!

Why is dang empty https://hn2.wilsonl.in/user/dang

What does the solid red dot for a username mean? Some have a green one and some do not have anything.

Good God I am a bore. May track some people down in my niche interest areas though, thank you for this.

Oi! Watch it! Our distance in embedding space is small. Then again, maybe I am a bore...

I find it surprising that I talk more about my hobbies than my work.

I guess I should apply for a job at ESA.

This vindicates my decision to comment using only short-lived burner accounts.

Wow I really like the topographic visualization of the data, looks really slick!

Thanks so much!

I wonder how high I am on the guns knowledge list?

I like how I am all over the place, haha.

I can already see the day coming when someone is going to make a phpBB-like HN skin which displays everyone's real name and LinkedIn picture along their comments.

Why not add their Gmail signature as well?

I was thinking to make that just now but as an opt in thing where you can upload your own avatar + banners + design your footer. I was also similarly worried about what is possible and what will be possible. With enough resources you can fingerprint writing styles and find who is who. That it can detect LLMs will be the excuse.

HN and other forums perpetuate a fallacy (that I assume isn't well known because nobody has published a research paper on it)... but "the wisdom of the crowd" isn't.

"Trusted voices" on HN are often not experts, yet espouse views which are not what the larger body of experts would consider correct. In addition, actual expert voices are drowned out by whatever the popular position is. HN also provides its own cultural bias, in that everything spoken on HN has to follow a rigorous cultural sieve set down by the guidelines, such that a "negative" view, even if correct, is considered either wrong or distasteful and buried. This is exacerbated by banning the use of humor to disarm controversial or heated comments. And this is the comments that HN does get; many opinions are never entered as comments here, so there exists a large knowledge gap. Then there's the "taboo" subjects like race, gender, religion, politics, social justice, etc which get buried for fear of controversy, so you're definitely not gonna find any expert opinions on those, as the stories just aren't there for discussion.

The end result is that often experts go unheeded or even downvoted, popular shallow opinions get upvoted, and substantive commentary based on evidence and experience is frequently missing. The fact is that we have no idea who knows what, or what's true or right. We just have "popularity" according to the particular cultural quirks of this site. So you can definitely find out "who thinks what", and who is considered to be more trustworthy to a HNer, but it has absolutely nothing to do with objective truth or the body of real knowledge that exists outside the world of HN comments. This is an echo chamber, but it's not a chamber of experts. It just seems that way because occasionally you see a minor tech celebrity, and people talk with absolute authority regardless of if they have any.

You want to find out who knows what? Look at their diplomas and careers. If they've done 20 years in a single field, probably they're an expert. If they have a degree (or multiple) in a field, probably they're an expert. If they spent half their life working on a single hobby, they're at least very knowledgeable in that field. But you can't determine that just by looking at who's talking about what or how many completely subjective "points" they get for what they say. Determining real knowledge requires analysis of specific criteria, filtered to get a higher quality result.

Humor is not banned on Hacker News. Bad attempts at it are penalized, though.

It's not "banned" but 90% frowned upon.

This is a serious site, people!

IMHO if there's any take away from the last 20 years of the internet, it's how hard it is to have decent communities of people online, curated like a blooming garden of diversity. What are some online communities that you appreciate?

I have a Ph.D. in neuroscience with exactly 20 years of experience. I talk about what I know and try to be curious about what I don't know. HN really helps me on both counts. How does it help you?

Searched for "Buttplugs".

My name pops up.

Hell yeah. A++++ completely accurate would use again.

Haha, now I must track down the contexts of buttplug conversations on HN.

Searching for neuropathy gives me results for psychopathy. Not exactly the same ...

Love it. Visited my profile, wasn't disappointed. 10/10

Searched for "dupe" and yes, it correctly pointed out the one user that takes it a hobby:

https://hn2.wilsonl.in/search/dupe

> Despite the intervening 16 years, we're amazed that social networks, even Hacker News, don't compute and display the trusted voices across topics

Please no. That sounds dystopian. We should not prefer having algorithms meddling with social interaction. We should not want things better designed to manipulate us.

May talk about social connections and knowledge networks etc, but this will eventually end up as a saas for everyones future HR. Can’t see much amazing here, especially with this founder’s vibe in the replies.

I guess it’s inevitable at this point, but how does it feel to be among the first who dumb down real people to a set of caricature keywords based on an ml method of the day?

> dumb down real people to a set of caricature keywords based on an ml method of the day

Better than at work where real people dumb down to a set of categories based on code, emails, age, gender, and projects?

Reply on news.ycombinator.com