Hacker News Reader

Comment by politelemon | Hacker News Reader

The crucial line:

> But again, there’s nothing to stop this. These subreddits can’t only fill themselves with joke articles to screw up a site like this, even if this one specific example is good for a laugh.

Most threads will be normal conversations, and reddit (and other discussion sites) can serve as a simple way of summarizing and generating news for 'free'.

I'm thinking that HN too could serve as a source for tech related news, couldn't it? Summarize the target article, then join it up with summaries/sentiments of the top comments in the thread. I didn't say I'm doing it, but if I could think of it, someone's probably way ahead of me already and has tried it.

View on HN

I keep hearing this sentiment on HN and IRL. As a journalist I think it misses the mark somewhat by failing to account for the value of reporting.

While some news can be generated exclusively from scraping Reddit threads or whatever, most decent journalism incorporates some form of reporting, i.e. the generation of novel information from trusted sources. Even without reporting, if you can't add to the store of knowledge in the world by writing the article, it doesn't offer any value to consumers or advertisers. That includes the the world of SEO spam. An effort has to be made to distinguish your work from the competition, or else your site isn't winning those top results.

Reddit threads are often just full of emotional responses to news already generated in this way. At some point along the line, a human has gone out and spoken to another human, forming an novel angle or argument, pursuing a line of inquiry, connected dots no one else has yet etc. That's news, not a summary of existing attitudes.

>I keep hearing this sentiment on HN and IRL. As a journalist I think it misses the mark somewhat by failing to account for the value of reporting.

There is valor added by journalists in even niche sectors. A journalist that reports on cars knows about the industry itself and can give an informed take on different developments, he might know how a car works, he might know about different trends in design, or markets, or whatever else. That is his added value.

When it comes to videogame journalism, though, they act as little more than spokespeople for corporations. They generally don't understand the product or how it works (mechanically or in terms of design), and in some cases aren't even adept at playing videogames themselves. The only thing the world would lose if no game journalist ever mentioned WoW again and the devs communicated directly with the playerbase would be the appearance of impartiality journalists give.

I am hardly a luminary of my field but if you take a look at some of my features you’ll see they are drastically different from the idea you may have in your head.

Lots of folks more accomplished than me who hit way harder. All the coverage about crunch, for instance, or sexual harassment scandals at big companies—-these are topics broken by games journos.

If your only exposure to this world is shitty SEO spam or cleverly-disguised marketing, I could see why you’d think the way you do. But there is so much good games journalism out there. I don’t recommend writing off the field like that.

That’s the problem, though, isn’t it? The vast majority of content that people see is low effort garbage that is pushed by media companies that are just looking to make a quick dime off of advertising revenue, and that’s very easy to do with AI.

Ad-based “news” is always going to have this problem, because getting clicks just requires getting people to the page; the quality of the content doesn’t matter nearly as much as the headline. The incentives for quality content just don’t exist in the current business model.

Paywalls obviously aren’t the solution, either, because news is nothing, now if it can’t be shared, and paywalls stop that dead. It also takes time and effort to build a brand, as you rightly stated before. The other important factor is how easy it is to lose trust in that brand, which means high stringent and transparent fact checking needs to be a part of the solution, alongside _proper_ retractions when someone _does_ get it wrong.

The only way I can imagine things improving is for independent journalists to get together to ditch the big media outlets and find some real solution to monetizing their work and keeping everyone accountable to accurate reporting. Obviously that’s much easier said than done.

> All the coverage about crunch, for instance, or sexual harassment scandals at big companies—-these are topics broken by games journos.

Sorry to be blunt but, that's not games. That's games journalists who want to be activists in their own fields. That, just as the gorillionth take on how this or that is problematic, is just people wanting to inject their politics into a hobby and it can't disappear soon enough.

I don't see that as an advantage of journalism. I see that as an evil to endure for a good that isn't there.

How is exposing malfeasance and illegal corporate activity activism?

I hate to retreat into platitudes here but good journalism shines a light in the dark, it comforts the afflicted and afflicts the comfortable, that’s kind of the name of the game.

If you’d prefer these topics go unaddressed and companies continue to take advantage of their workers or whatever, not sure we have enough in common to have real discourse on the subject. Sorry to be blunt lol.

>All Game Journalists do is act as little more than spokespeople for corporations

>>Actually here Game Journalists also report on X, Y and Z

>Nah that is just people injecting politics and being activists

This is a pretty dumb "No true Scotsman" argument you are smuggling into the conversation here.

> As a journalist I think it misses the mark somewhat by failing to account for the value of reporting.

At a revenue level how do you quantify that value? Unless you are a brand (TIME, Washington Post, etc) the only metrics are page views and time spent reading the article that eyeballs wander across ads.

Arguably a clear well written article may perform worse than an AI generated article that bleeds over on to a second page. Some of the most valuable content on the internet is not well written pieces on the political climate, but "OMG YOU'LL NEVER GUESS HOW THIS CHILD CELEB TURNED OUT" with a 75+ image one per page photo gallery.

Edit: made a typo.

How do I quantify the value of reporting? As a writer and reporter, not a finance dude, I'm really not qualified to answer that. But it's interesting you say 'unless you are a brand' because I feel like the brand is what you get from building up years and years of trust with excellent reporting.

In your second paragraph, I don't think your assertion is correct. It isn't 2008. While you were correct at one time based on trends in the industry, "OMG YOU'LL NEVER GUESS HOW THIS CHILD CELEB TURNED OUT" content never cemented itself. It had a high-water mark and that kind of thing hasn't made good money in years. The New York Times is still the most valuable brand in journalism and it's thriving against its competition. Not only other legacy brands, but let's be real, look where Buzzfeed is at today. Shitty content lost revenue-wise as well as on a moral basis. The only way to reliably make good ad revenue is to spam articles but it's not a profitable (or serious imo) way to run a journalistic enterprise. I believe paywalls/subscription models will continue to dominate while the losers fight for scraps.

> While you were correct at one time based on trends in the industry, "OMG YOU'LL NEVER GUESS HOW THIS CHILD CELEB TURNED OUT" content never cemented itself. It had a high-water mark and that kind of thing hasn't made good money in years.

The LA Times makes $380,116 per employee. Outbrain, one of the largest click farms makes $841,768 per employee. Taboola makes $784,780 per employee. Combined the latter two bring in enough revenue to equal 1/4 of the entire print media industry. That isn't even counting hybrid companies like media.net that do a combination of clickbait and traditional advertising.

(This is all 2022 data)

Hey sorry, I thought we were talking journalism. Outbrain is an advertising/recommendation engine company right? Know zero about them but doesn’t look like they produce anything. Interesting you went with revenue per employee though because I think market cap tells a different story. They’re worth 256 million today and NYT is sitting at 6 billion. I don’t doubt clicky clicky makes money, just that brands in journalism that rely on clicky clicky don’t make as much as those that produce years and years of high-quality content.

I return to Buzzfeed because I think they’re a good example of a place that tried to do both but because they had so much low-brow clicky clicky, the brand ultimately suffered. Despite winning a Pulitzer in 2021 it really has a terrible reputation among non-industry folks who haven’t forgotten how the brand came to prominence.

That was specifically my point. You can't quantify any value gain of good journalism. Hiring a 10% better writer doesn't earn you 10% more money. I quoted the revenue per employee numbers because they highlight the fact that hiring another engineer to build better click bait farms will earn you more than hiring another writer.

Don't get me wrong, I appreciate good journalism. 90% of the world does not and they just want to be spoon fed garbage at a faster rate. Look at the most popular shows on TV for example, they aren't National Geographic documentaries, but instead trashy reality television.

As a common man someone who hates what journalism has become: I hope "AI" swiftly replaces your industry which has become a cancer and plague upon society. I would rather deal with a used car salesman than a journalist.

Journalists today only provide unnecessary exposition and emotional poetry (read: wasting my reading time) and actively cause more problems and conflicts in society than they address and resolve (sensationalism and fearmongering is what brings in the clicks and thus the money).

I have no sympathy whatsoever for such a rotten, morally devoid, worthless industry, and while I doubt "AI" will bring about any fundamental difference or improvement it will at least make the process of journalism better reflect the actual value of the final product.

I think you might be reading the wrong journalistic publications.

You are commenting on an article written by a journalist. :)

In case it wasn't obvious, I didn't bother reading or even clicking on the article. Not the least because it comes from the cesspool known as Forbes, of all places.

Was going to say the same thing "as a common woman" about the SV turbo-capitalist mindplague that's swept across the world!

I created a website that does the following:

- pick HN items with more than 400 votes

- gather their titles in a list

- ask the AI to filter out the ones that are not tech-related (bay-area topics, politics)

- scrape selected articles

- write summaries

- publish static website

other sources are reddit subreddits and rss feeds (official languages' blogs and github's releases page). The AI is quite gullible. That can be avoided by giving it more context and having a review step where you make sure that you are enforcing your editorial rules. Another thing that I've been wondering is having a cheaper model (gpt-turbo-3.5) write articles and then use a more sophisticated to review them (gpt4)

> I created a website

I’d really like to see this - any chance we could?

It's https://dev-radar.com/, the repository is https://github.com/lfarroco/news-radar

Very difficult to read on mobile.

that's true! added some responsive classes to the columns

Thanks, looks much better now!

This is really neat and way easier to read.

I’d like the inverse too (the rejected links! Politics, history etc).

They are a great source of weekend reading. But you’ve included the repo, so now that’s on me.

It should be possible to adjust the prompt asking the ai to include a reason for inclusion/rejection too.

This weekend I'm going to make it easier to run customized instances. I wasn't expecting a good reception from folks lol

some additional info: this is the prompt that filters "candidates" to be scraped: https://github.com/lfarroco/news-radar/blob/main/src/candida...

This is the prompt that generates the articles: https://github.com/lfarroco/news-radar/blob/main/src/writer....

I'm curious, can you ballpark how many tokens you use for the round trip of link ingestion and article creation? I haven't really tinkered with LLMs, so I'm trying to wrap my head around the cost of projects like this.

I run the script every 6hrs. When a candidate list is built, it has around 15 items + the prompt. That usually goes around 1k tokens. When generating an article, the prompt has 1k tokens and the response has around 1k tokens (I ask for text with up to 250 words). If the source article is long I just truncate it - it should be enough to create a summary presenting the topic. It is possible to batch article creation, submitting the prompt once alongside 2-articles, but I don't think that it would be worth the hassle. I'm getting billed 1c for every 12-15 articles generated (using gpt3.5turbo

I particularly appreciate that you have a disclaimer at the top stating that the article was AI generated. I wish more sites that used AI did that. Nice execution too.

Even if you don't want to just summarise the comments, you can view the comments section for most articles as crowd-sourced research on the topic. You can easily walk away from most interesting discussions here with a shortlist of topical articles, books, etc, and often some colour commentary straight from the horse's mouth.

Anon because I know startups working on that exist and should not talk about it

Information wants to be free; right now it’s a browser plugin that reads DOM and allows tagging; that human work is fed into a training system

The idea started as make Twitter community notes it’s own thing as browser plug-in and then expanded into model training to look like a user browsing the web and read DOM

Not hitting an API, so no fees yes? It’s the user making the interaction choice

Rate limits are going to be the only defense against AI trained to look like doom scrollers

They can't only do that but they can embarrass someone who's passing an AI writer off as human. At minimum, they'll have to message in some way that the article was written by AI lest this trick is pulled to embarrass an author which was otherwise considered to be human.

Reply on news.ycombinator.com