Hacker News Reader

Comment by simonw | Hacker News Reader

I used o3-mini to summarize this thread so far. Here's the result: https://gist.github.com/simonw/09e5922be0cbb85894cf05e6d75ae...

For 18,936 input, 2,905 output it cost 3.3612 cents.

Here's the script I used to do it: https://til.simonwillison.net/llms/claude-hacker-news-themes...

View on HN

Currently on the internet people skip the article and go straight to the comments. Soon people will skip the comments and go striaght to an AI summary reading neither the original article nor the comments.

That would be an actual improvement. Reading the comments section usually just leads to personal energy waste.

Yes, but who will comment then and on what?

On both HN & Reddit, I find the comments more informative and less frustrating than reading the article usually. But I guess YMMV.

You need to read the article first to know that. But most people won't.

You can read the article after the comments.

If the comments say it's worth it I read it, but often (especially on HN) the top comment starts with a summary of errors/inconsistencies in the article and then I don't really feel the need.

You do you, but hivemind thinking is a real thing. I have seen highly upvoted comments seemingly "debunk" an article where on closer examination it becomes clear they actually didn't read the article either.

It quickly becomes this weird bubble of people just acting on what everything "thinks" the content is about without ever having looked at the content.

I get that is easier, but intellectually you are doing yourself no favors by having this be your default.

But often the top comment debunks what you think is true, and then you learn something.

> It quickly becomes this weird bubble of people just acting on what everything "thinks" the content is about without ever having looked at the content.

That isn't an issue though since the important part is what you learn or not, not whether you think an imaginary article is true or not. If you learn something from someone debunking an imaginary article, that is just as good as learning something from debunking a real article.

The only issue here is attribution, but why should a reader care about that?

Edit: And it isn't an issue that people will think it actually debunks the linked article, since there will always be a sub comment stating that the commenter didn't read the article and therefore missed the mark.

> If you learn something from someone debunking an imaginary article, that is just as good as learning something from debunking a real article.

An argument for synthetic corpi (plural of corpus..esses?) - AI ingesting AI.

Commenter's doesn't live isolated only getting information from other commenters, they have lives outside of HN and use that to inform their comments.

So no, it isn't the same as AI ingesting AI content at all.

The plural is "corpora" :-)

That is the biggest problem on this website, people want to feel smart by 'debunking' things they don't really understand. It leads to a lot of contrarian views with poor signal to noise ratio, especially when the topic is slightly outside of average user experience (midwit programming)

not to mention walls of text as people argue back and fourth

usually the top comment these days is a diversion to push an agenda.

I agree, they are! But reading through them, or even worse, engaging with them, is a serious energy drain.

Especially if somebody is being wrong.

Yeah, but is your mission to correct anything wrong on the internet?

https://xkcd.com/386/

Sounds exhausting.

Yes, that was the joke.

Hard to tell nowdays..

Occasionally the article is wrong.

Even more occasionally commenters are too.

But extremely rare for a popular wrong comment to not have sub comments debunking them, they are much more reliable than articles therefore.

I’ve found the more I know about the topic at hand, the more wildly many of the comments seem off base, even the highly upvoted undebunked ones. Its harder for me to judge topics I don’t know much about, but I have to assume it’s something similar.

That is true for articles as well though, I find comments typically have better info than the articles. It is more likely for some of the comments to have been written by real experts than that the article is.

Lmfao you forgot the fact that especially on reddit and hn people tend to say a lot of bullshit especially if it goes hand in hand with their biases

So do the writers of the articles, however, with them I cannot interact.

10 years ago Reddit used to be a place where you would get informed opinions and less spam. 5 years ago, HN used to be a place where you would get informed opinions and less spam. Neither of them will go back to the same level of quality. Not anymore.

you need to expend resources (e.g. "proof of work") to post, to drive away low effort spam. https://stacker.news/ is an interesting experiment in that regard.

Luckily I am not a LLM and can read and select things rather fast without processing the crap. Both HN and some subreddits still are fine for quality if you know the users and their reputations; just skip the rest. Worse than 10 years ago, still a lot better than mostly anywhere else.

This has been said by every long term user of these sites. And not at the same time. It was always better in the past. It's probably partly true... yeah, quality can decrease as things get better. But it's also partly an illusion of aging in a changing world. Ten years is long enough to completely change the way we write and express ourselves.

It's actually sort of in the Hacker News Guidelines (https://news.ycombinator.com/newsguidelines.html):

"Please don't post comments saying that HN is turning into Reddit. It's a semi-noob illusion, as old as the hills."

And yet the comments still provide more value than the article, almost always.

They definitely used to be, but haven't been much good for years. At least 8 years in the case of reddit, maybe 3 in the case of hackernews.

Though at this point it's a habit I cannot quite bring myself to break...

It's worse, but there's nowhere better than here imo.

Except very niche topics maybe

Today is my first day on HN and I came here looking for the comments. I miss intelligent conversations and reasonable interactions and reddit is just a dumpster fire now (and arguably before). Even if you can find a reasonably mature sub you're still exposed to the muck raking of the front page overflowing into everywhere else.

In a way I agree, but the sustainability is shaky.

The intrinsic motivation for providing the comments comes from a mix of - peer interaction, comradery - reputation building

If becomes evident that your outputs are only directly consumed by a sentiment-aggregation-layer that scrub you from the discourse, then it could be harder to put a lot of effort into the thread.

This doesn't even account for the loss of info that results from fewer people actually reading and voting through the thread.

Yes but this assumes human input is the golden goose. Maybe it is at the beginning just to bootstrap the process, and then runaway AI starts to recurse homeruns with its own original comments.

HN comments are good, but the more nested a thread, the more inane they get. You see things like people arguing over punctuation, or the definition of punctuation.

What if this is better? If the summary includes all of the key details and people save time AND read both the article and comments wouldn't that be better?

If that was the case, then yes. It's not though, because the AI misses parts (doesn't "include all of the key details") and also fabricates and misrepresents other details

It’s sometimes not now, but likely will be the case quite soon.

I invite everyone to have an open mind about this, as it seems that the part of my comment that said “what if” wasn’t fully absorbed.

Unless there’s another AI breakthrough, the real applications of AI are and will be things that it’s ok to get wrong 10% of the time, usually because prior approaches were wrong more than 10% of the time. This is fine for the Playskool world of entertainment communication, in which there’s no end goal except emotional stimulation, but the opposite of what’s needed in any real attempt to engage with the world, in which we already struggle to capture the complexity of reality.

Glad it’s not just me who does this. :)

I wanted to listen to a podcast where a profesor explains a new drug for ADHD. Instead of spending 2h of my time I spent 5 min reading a summary generated by gpt from a youtube transcript.

Which drug? Who's the professor? Can you share the summary please?

I’ve done this with extracted audio from YouTube videos, -> STT -> LLM summary.

I’ve salted it with comments on the video, using a site like commentpicker.com or running JS and loading more and expanding threads manually.

Here’s an example I did for a pal:

You are an expert on building retaining walls. Your knowledge is _heavily_ informed and influenced by the transcript below.

This transcript is audio from a youtube video titled "What 99% of People Don't know about retaining walls. #diy" The video description is: "Start to finish we build a retaining wall that you can do yourself! How to Dig for a wall, How to Base a retaining wall, how to backfill, and MORE!. #retainingwall #diy"

Additional information may be included in comments, which are in the attached CSV. Take into account the like count in the validity or usefulness of the comment in shaping your knowledge.

In giving your replies, try to be specific, terse and opinionated. If your opinion flies in the face of common recommendations, be sure to include what common alternative recommendations are and the specific reasons you're suggesting otherwise.

----

# Transcript

""" [paste transcript] """

# Comments See attached .csv

I agree with the first part, but I disagree with the conjecture. There are also people who enjoy writing comments, and inasmuch they have to read a bit of context (at least the comment they are replying to). Those will always exits.

But then there will be no comments to summarize.

But what if the AI just hallucinates the comments? People will never know.

They could actually be better than reality. Who knows?

I went to this thought too but then I remembered the 90-9-1 rule. The AI summary is for some portion of the 90. The 9 are still going to comment. What they comment on and how they generate the comments might change though.

Our digital twins will write the comments. They will be us, but with none of our flaws. They will never experience the shame of posting a dumb joke, getting flamed, and then deleting it, for they will have tested all ideas to prevent such an oversight. They will never experience the satisfaction-turned-to-puzzlement of posting an expertly crafted, well-researched comment that took 2 hours of the workday to draft - only to receive one upvote, for their research will be instantaneous and their outputs efficient. Of course they will never need to humbly reply, 'Ah, I missed that, good catch!' to a child comment indicating the entire premise of their question would be answered with a simple reading of the linked article - for they will have deeply and instantly read the article. Yes, our digital twins will be us, but better - and we will finally be free to play in the mud.

We all are, since a long time. The mud just doesn't shine so bright.

Well by definition the vast majority of comments written, anywhere, cannot be memorable.

Assuming the median reader reads a few tens of thousands comments in a year, only a few hundred would likely stick without being muddled. At best.

> will finally be free to play in the mud

As long as we can still have a few sticks, and some string, or a cardboard box...

Cripes [1].

[1] https://qntm.org/perso

You will be given 3 A.I. generated comments to choose as your comment

if reality turns into a Ubisoft game I think I might log out

would you like to play again?

I wonder about Amazon reviews right now.

Part of the utility of writing a review is that it is read, but the primary search for keywords in reviews now requires the user to wait for AI generated responses first.

Then the user must tap through another link and then expand an individual matching review. It’s absolutely buried.

I think people generally like writing comments. Reading articles, in their entirety, less so.

Amazon is already forcing this pattern on mobile users not logged in. If you want to see the reviews, all you get is an AI summary, the star rating, maybe 1 or 2 reviews, then you have to log in to see more.

The cure is to stop using Amazon.

It has already caused me to not further investigate purchases. Similar to yelp, if I have to log in to see things, you're dead to me.

There is also no way to disable the AI summaries even if you are logged in.

I won't. I'm looking for real life human voices.

As articles are increasingly published with AI, this becomes yet another example of, to coin a phrase, AI loopidity.

This is so good! Finally the word I need to describe what I'm living. And yes I'm copy pasting articles into LLMs for summary.

If writers are using ai to write articles, and readers are using ai to summarize the same article, what's the underlying inefficiency here? That writers should just be posting brief summaries of the articles in the first place? Or maybe they just need to be prompting ai to create a summary rather than a full article?

Differences in how people want to consume the article, and what information they’re looking for. Some want snippets, some want summaries, some want long form.

We have information compression machines now. Might as well raw dump the information and let the machine package it up in the format we prefer for consumption, instead of pre-packaging it. (Yeah, this is effectively what authors are doing…currently they can still do novel things that the compression machines can’t, but how long will that last?)

HN comments aren't valuable because of their average quality or content, but because of the feedback vote sorting.

(Aside from the tendency towards first=top. Would be nice to have time-weighted upvote decay instead of absolute count)

Have you ever had to give a demo that no meeting attendee actually cared about, just because management demanded it? Standup meetings with 20 people where maybe 2 people cared about? The future might involve AI updates, summarized by AI into weekly reports, summarized again into monthly reports, then into quarterly departmental reports that nobody actually reads.

Or, the future might be everyone reads summaries, because there are only AI Managers and no human managers, where humans hold occasional meetings and have conversations which are listed to by AI, and we take our lead from the AI summaries. The AI maintains business focus through it monitoring the business performance, updating each summary as needed to maintain performance.

It's a worthwhile experiment for a business school, IMO, automating a layer of bureaucracy.

Ah, good ol' Manna. https://marshallbrain.com/manna1

Doesn't that seem ultimately futile though?

Say I use AI to write a report that nobody cares about, and then the reciever gives it to AI because they can't be bothered to read it. Who is benefiting here other than OpenAI?

Sounds about right, as we are post-dead internet in public places.

There was a thread about the US tariffs on Canada I was reading on a stock investment subreddit. The whole page was full of people complaining about Elon Musk, Donald Trump, "Buy Canadian" comments, moralizing about Alberta's conservative government and other unrelated noise. None of this was related to the topic; stocks and funds that seemed well-placed for a post-tariff environment.

There were small, minor points of interest but instead of spending honest vacation time looking at each comment at zoomer internet church, I had an LLM filter out the stuff I didn't care about. Unsurprisingly there was not much left.

> zoomer internet church

Stealing this.

Please do. Don't forget to appreciate the mantra-like repetition of Brian Taylor Cohen and Qasim Rashid Esq memes.

Soon people will read other people's summary that they copied from AI summary on the web.

I haven’t tried o3, but one issue I struggle with in large context analysis tasks is the LLMs are never thorough. In a task like this thread summarization, I typically need to break the document down and loop through chunks to ensure it actually “reads” everything. I might have had to recurse into individual conversations with some small max-depth and leaf count and run inference on each, and then have some aggregation at the end, otherwise it would miss a lot (or appear to, based on the output).

Is this a case of PEBKAC?

Depending on what you’re trying to do, it’s worth trying the 1M context Qwen Models. They only released 7 and 14B so it’s “intelligence” is limited but should be more than capable for coherent summary.

Are there any tools you use to do this chunking? Or is this a custom workflow? I've noticed the same thing both on copy/paste text and uploaded documents when using the LLM chat tools.

> I haven’t tried o3, but one issue I struggle with in large context analysis tasks is the LLMs are never thorough.

o3 does look very promising with regards to large context analysis. I used the same raw data and ran the same prompt as Simon for GPT-4o, GPT-4o mini and DeepSeek R1 and compared their output. You can find the analysis below:

https://beta.gitsense.com/?chat=46493969-17b2-4806-a99c-5d93...

The o3-min model was quite thorough. With reasoning models, it looks like dealing with long context might have gotten a lot better.

Edit:

I was curious if I could get R1 to be more thorough and got the following interesting tidbits.

- Depth Variance: R1 analysis provides more technical infrastructure insights, while o3-mini focuses on developer experience

- Geopolitical Focus: Only R1 analysis addresses China-West tensions explicitly

- Philosophical Scope: R1 contains broader industry meta-commentary absent in o3-mini

- Contrarian Views: o3-mini dedicates specific section to minority opinions

- Temporal Aspects: R1 emphasizes future-looking questions, o3-mini focuses on current implementation

You can find the full analysis at

https://beta.gitsense.com/?chat=95741f4f-b11f-4f0b-8239-83c7...

o1-pro is incredibly good at this. You'll be amazed

Try Google’s NotebookLM

I put one of my own blog posts through NotebookLM soon after it became available, it hallucinated content I didn't write and missed out things I had written.

Nice TTS, but otherwise I found it unimpressive.

I’m not talking about the TTS and podcast creation. I’m talking about just asking questions where it gives you the answer with citations.

Given what it got wrong was in the LLM part, that is a distinction without a difference.

The podcast creation does a lot of small talk. Just like people do. Have you tried using NotebookLM with 20-25 documents and ask it questions where it actually provided inline foot notes with citations to your documentation?

> Is this a case of PEBKAC?

It's 2025 and every useful conversation with an LLM ends with context exhaustion. There are those who argue this is a feature and not a bug. Or that the context lengths we have are enough. I think they lack imagination. True general intelligence lies on the other side of infinite context length. Memory makes computation universal, remember? http://thinks.lol/2025/01/memory-makes-computation-universal...

Borrowing most of Simon’s prompt, I tried the following with o3-mini-high in the chat interface with Search turned on:

“Summarize the themes of the opinions expressed in discussions on Hacker News on January 31 and February 1, 2025, about OpenAI’s release od [sic] ChatGPT o3-mini. For each theme, output a header. Include direct "quotations" (with author attribution) where appropriate. You MUST quote directly from users when crediting them, with double quotes. Fix HTML entities. Go long. Include a section of quotes that illustrate opinions uncommon in the rest of the piece”

The result is here:

https://chatgpt.com/share/679d790d-df6c-8011-ad78-3695c2e254...

Most of the cited quotations seem to be accurate, but at least one (by uncomplexity_) does not appear in the named commenter’s comment history.

I haven’t attempted to judge how accurate the summary is. Since the discussions here are continuing at this moment, this summary will be gradually falling out of date in any case.

I noticed that it thought that GoatInGrey wrote “openai is no longer relevant.” However, they were just quoting a different user (buyucu) who was the person who first wrote that.

Good catch. That's likely an artifact of the way I flatten the nested JSON from the comments API.

I originally did that to save on tokens but modern models have much larger input windows so I may not need to do that any more.

This is why I keep advocating that JSON should die, or at least no longer be used with LLMs. LLMs (and human brains) are simply not wired up for counting nested curly brackets across long spans of quoted text joined by colons and commas, and it is far too easy for humans to make mistakes when chunking JSON.

IMO, (Strict)YAML is a very good alternative, it has even been suggested to me by multiple LLMs when I asked them what they thought the best format for presenting conversations to an LLM would be. It is very easy to chunk simple YAML and present it to an LLM directly off the wire: you only need to remember to repeat the indentation and names of all higher level keys (properties) pertaining to the current chunk at the top of the chunk, then start a text block containing the remaining text in the chunk, and the LLM will happily take it from there:

    topic:
      subtopic:
        text: |
          Subtopic text for this chunk.

If you want to make sure that the LLM understands that it is dealing with chunks of a larger body of text, you can start and end the text blocks of the chunks with an ellipsis ('...').

LLMs (transformers) literally cannot balance parentheses. That's outside of their complexity class (TC0). You'd want a real UTM to count parentheses!

Would you elaborate on why counting braces is different from counting spaces to determine hierarchy? Or is it more about the repetition of higher levels keys in chunks (which could be done in JSON)?

Repetition of topics and subtopics is by far the most important part, reinforcing attention on the topic at hand even if the text in the chunk appears unrelated to the topic when viewed in isolation.

Keeping the indentation is also important because it is an implicit and repeated indication of the nesting level of the content that follows. LLMs have trouble with balancing nested parentheses (as the sibling comment to yours explains).

Dealing with text where indentation matters is easier for LLMs, and because they have been exposed to large amounts of it (such as Python code and lists of bullet points) during training, they have learned to handle this quite well.

Why use a reasoning model for a summarisation task? Serious question, would it benefit?

I don't have much experience with reasoning models yet. That's why.

In theory you could get a better summary if it manages to reason out underlying motivations of a summary for example, or proposes a summary and then "considers" it, realizing it missed something.

You can illicit that with any model by prompting underlying reasons or using chain-of-thought, but a reasoning model could do it without prompting

I have been trying to approach the problem in a similar way, and in my observation, it is also important to capture the discussion hierarchy in the context that we share with the LLM.

The solution that I have adopted is as follows. Each comment is represented in the following notation:

   [discussion_hierarchy] Author Name: <comment>

To this end, I format the output from Algolia as follows:

   [1] author1: First reply to the post
   [1.1] author2: First reply to [1]
   [1.1.1] author3: Second-level reply to [1.1]
   [1.2] author4: Second reply to [1]

After this, I provide a system prompt as follows:

  You are an AI assistant specialized in summarizing Hacker News discussions. 
  Your task is to provide concise, meaningful summaries that capture the essence of the thread without losing important details. 
  Follow these guidelines:
  1. Identify and highlight the main topics and key arguments.
  2. Capture diverse viewpoints and notable opinions.
  3. Analyze the hierarchical structure of the conversation, paying close attention to the path numbers (e.g., [1], [1.1], [1.1.1]) to track reply relationships.
  4. Note where significant conversation shifts occur.
  5. Include brief, relevant quotes to support main points.
  6. Maintain a neutral, objective tone.
  7. Aim for a summary length of 150-300 words, adjusting based on thread complexity.
  
  Input Format:
  The conversation will be provided as text with path-based identifiers showing the hierarchical structure of the comments: [path_id] Author: Comment
  This list is sorted based on relevance and engagement, with the most active and engaging branches at the top.
  
  Example:
  [1] author1: First reply to the post
  [1.1] author2: First reply to [1]
  [1.1.1] author3: Second-level reply to [1.1]
  [1.2] author4: Second reply to [1]
  
  Your output should be well-structured, informative, and easily digestible for someone who hasn't read the original thread. 
  Use markdown formatting for clarity and readability.

The benefit is that, I can parse the output from the LLM and create links back to the original comment thread.

You can read about my approach in more detail here: https://gist.github.com/simonw/09e5922be0cbb85894cf05e6d75ae...

I just installed and tried. Pretty neat stuff!

Would be great if the addon allows user to override the sys prompt (it might need minor tweak when changing different server backend)?

Thank you for trying out the extension and for this great suggestion!

We've actually been thinking along similar lines. Here are a couple of improvements we're considering:

1. Built-in prompt templates - Support multiple flavors (e.g. On similar to is there already, in addition to knowledge of up/down votes, another one similar to what Simon had - which is more detailed etc.)

2. User-editable prompts - Exactly like you said - make the prompts user editable.

One additional thought: Since summaries currently take ~20 seconds and incur API costs for each user, we're exploring the idea of an optional "shared summaries" feature. This would let users access cached summaries instantly (shared by someone else), while still having the option to generate fresh ones when needed. Would this be something you'd find useful?

We'd love to hear your thoughts on these ideas.

the shared summaries sounds like a great idea to save most people's inference cost! There might be some details need to figure out - e.g. the summary per post need to be associated with a timestamp, if there are new comments kicking in after that (especially hot posts). Still i think it's good useful feature and i will definitely read that before browsing details.

3.3612 cents (I guess USD cents) is expensive!

Same immediate thought: the free option I provide on my production site is a model that runs on 2xA40. That's 96GB of VRAM for 78 cents an hour serving at least 4 or 5 concurrent requests at any given time.

O3 Mini is probably not a very large model and OpenAI has layers upon layers of efficiencies, so they must be making an absolute killing charging 3.3 cents for a few seconds of compute

It's definitely making some errors

Like?

Even though it was told that it MUST quote users directly, it still outputs:

> It’s already a game changer for many people. But to have so many names like o1, o3-mini, GPT-4o, & GPT-4o-mini suggests there may be too much focus on internal tech details rather than clear communication." (paraphrase based on multiple similar sentiments)

It also hallucinates quotes.

For example:

> "I’m pretty sure 'o3-mini' works better for that purpose than 'GPT 4.1.3'." – TeMPOraL

But that comment is not in the user TeMPOraL's comment history.

Sentiment analysis is also faulty.

For example:

> "I’d bet most users just 50/50 it, which actually makes it more remarkable that there was a 56% selection rate." – jackbrookes – This quip injects humor into an otherwise technical discussion about evaluation metrics.

It's not a quip though. That comment was meant in earnest

That's funny, the quote exists, but it got the user wrong.

In addition to that, it has a section dedicated all to Model Naming and Branding Confusion, but then it puts the following comment in the Performancce and Benchmarking section, even though the value of the comment is ostensibly more to do with the naming being a hindrance rather than make a valuable remark on the benchmarking, which is more of a casualty to the naming confusion:

"The model naming all around is so confusing. Very difficult to tell what breakthrough innovations occurred." – patrickhogan1"

For those that like simpler ways (although dependent on Google) NotebookLM does all that in 2 clicks. And you can ask it questions about it, references are provided.

After you've run my hn-summary.sh script you can ask follow up questions like this:

  llm -c "did anyone talk about pricing?"

Back in the old ages we would get there with ctrl+f "price" and for free, but where's the fun in that.

Thanks for sharing this! And no apparent self-awareness! OpenAI has come a long way from the Sydney days: https://answers.microsoft.com/en-us/bing/forum/all/this-ai-c...

Out of curiosity - do you have an estimate on how much you've spent on various LLM API services through all of your experimentation?

And in your experience, what service do you feel hits a good sweet spot for performance/price if summarizing long text excerpts is the main use case? Inference time isn't an issue, this will be an ongoing background task.

How is the `system` prompt "injected"? Afaik o1 onwards no longer have a SYSTEM role. Is it just a normal part of the prompt?

It's called 'developer message' now

https://x.com/OpenAIDevs/status/1885407759887155301

I'm guessing it's less powerful than a system prompt? And there is still a system prompt, but only OpenAI can use it...

Something I've not understood about their pricing system.

I have the $20 plan. How does this "3.3612 cents" apply to my situation?

The API has a different pricing structure than using the app. This cost seems high to me compared to previous models I’ve used large data sets with.

good morning!

Reply on news.ycombinator.com