Google cut a deal with Reddit for AI training data
theverge.com
theverge.com
Arguably Reddit's value is it's data, and GOOG is renting it for 1.2%/year?
2. It isn't an exclusive license. There are dozens of companies training language models, and if they are all forced to pay the same amount then that's some serious cash.
Mozilla being an exception, I suppose?
But 80+% of their revenue, and probably the highest margin bit of revenue, comes from that Google payment. So whatever their value is, hinges on that contract payment. Thus, the multiple is interesting....where you might "compare total company value with an annual payment" to see what that mathematical relationship is. Especially given their rapidly declining market share.
Otherwise, Reddit is productizing its data, which was the obvious next step after closing public API access. This establishes a market price that at least one buyer was willing to accept, and shapes the terms for later deals with more customers.
Presumably, they think they can sell this kind of access many time over as one of their major revenue channels while also benefiting from the incoming traffic it should generate.
I imagine this wont be the last announcement in this space.
Reddit has a chicken and egg problem - nobody knows what the value of it's data even is. They've never sold it before. You can sell pet rocks for 1 billion dollars, but nobody will value them if there's been 0 sales. (ask vs bid vs last sale price).
I'm guessing reddit is rushing to sell their data so they can show that their data is worth anything at all.
Do we know this?
It's possible they've sold it and we haven't heard about it since then. I would think they'd want to brag about it publicly though.
B. Reddit's value is in the community more than the data. A company with Reddit's data but not Reddit's users is worth relatively little.
I’m not in AI research so I know I could be completely off base.
The only things I can think of Google using that data for are for nefarious purposes.
My usage went completely downhill and my posts/comments as well
This is just adding in more recent content.
> gets a pedantic reply
were you also trained on reddit?
Reddit is ok and quite cool for targeted discussion on targeted sub-reddits. But all the general subreddits visited by general population and everything that pops once in a while on the front page is a target for hive mind.
For HN comparison, there is a lot of "wrong" here too, but here you can find a cited academic study from one good American university that reveals most of the botfarms and fake news disseminators come from western sphere. If you try to claim on Reddit or anywhere on the internet that fake news champion is not Russia+China+whoever is evil, your entry will get buried.
Also, ask yourself who's the median redditor. For my country's national subreddit the median redditor is a high school kid from the capital.
If there is a thing, and topic about it has reached the internet discussions, the participants will have an opinion regardless of their actual practical knowhow about the thing. They'll form their opinion otherways. Platforms like Reddit favour a master opinion due to score and moderation system, so one out of N wrong theories will surface as the master opinion.
In real life, if you ask bunch of random people about the thing they don't know, you get wildly different answers, and largely no-one will back up anyone there. Certainly not in enough force to push a confidently wrong answer up as the "people's opinion"
This must be the real reason for destroying public API access.
Reddit is IPOing next month. IMHO they cut public API access as a way to simplify and pump bookkeeping numbers to get a better listing price.
Also how many of the posts are already generated by AI? Seems like a raw deal to pay for data that could be a large percentage of bot activity.
Why visit Reddit at all for information when you can just ask Gemini.
See this directive in https://www.reddit.com/robots.txt `Disallow: /r//comments////*`
But I can still search Reddit on Google. How does Google manage to get the data?
The relevant robots.txt rules are:
User-Agent: *
Disallow: */comment/*
Disallow: /r/*/comments/*/*/*/*
The url for the comment sections looks like: https://www.reddit.com/r/[subreddit]/comments/[id]/[slug]/. This doesn't match the above rules because there's only 2 parts after "comments", not 4 parts as specified by the rules.I question whether Reddit data is worth anything at all.
On the other hand they bought a poison pill
Is whoever behind the decision making here blindsided by Reddit being a massive trove of data, and are not realizing most of that data is shit?
And that's a sweet deal for Google, paying pennies to partially patch their deteriorating reputation about their search engine becoming useless.
I figured they should pay for it. But now that they do (in this instance), I'm realizing this might be even worse. They can just buy the entire market in the same way they buy Google search users by paying Apple billions a year.
And still the actual content creator, a Reddit user in this case, is not compensated.
It's truly wild how lax regulation is. This is probably the most important technology ever created and we just let 2 companies have it all: the data and the compute.
I'm not saying we should do it, but it's a fascinating thought.
I wrote an essay on this here: https://magis.substack.com/p/llm-data-sales-a-market-for-lem...
They will helping Google create a real-time pipeline to Reddit's data.
But that's the future we're in, it's not gonna get rolled back.
Everyone is privacy focused until they're not
I think the entire thing is scummy too, but its interesting to me how people think that advertisers were somehow less bad than this.
The article was about mining comments for AI, not about advertisement. Yet you brought in the topic of advertisement, and then created a fake argument about how people think advertisers are better than AI scraping. It's really weird that you did that.
I would love to see a leak of that dataset, sorted my username.
If people on reddit wanted to get paid for their opinions, they wouldn't be posting them for free on a publicly accessible forum that does not even require an account to view...
This same sentiment came up when StackExchange started to use/sell StackOverflow posts for AI training too - yet everyone arguing that position failed to recognize StackExchange has been making money off their free contributions since day one. Like, how else would StackExchange even exist? They can sell job postings now because of all the free contributions and community built up over the past decade - same with Reddit...
https://www.brentozar.com/archive/2015/10/how-to-download-th...
People post on the internet 'for free' with the expectation that they're discussing ideas which have their own value, not providing training data. Yeah yeah ToS blah blah blah, but the goodwill that sites have built – even ones like HN – is on the premise that we're participating in the marketplace of ideas, not enriching shareholders.
This will ultimately be bad for the open internet.
The only reason Reddit is interesting to advertisers is the community. You can select specific subs to advertise to, interest groups, demographics, etc. You can get "laser" focused on who you advertise to, and offer compelling adverts to people highly likely to engage with your products/brands and convert into sales.
The only reason this works is because people like you have built those communities on Reddit by contributing your thoughts, opinions, experiences, expertise, time and more for free.
I'm not fine with them using that intellectual labor for any purpose whatsoever, indefinitely, forever. The only way to win at this point will be not to play; the downstream consequences will be the death of the open internet.
It's a trade: y-combinator, by cultivating a group of talented technical contributors, gets to advertise hiring posts for free and, in some ways, autistically contributes to the community.
In exchange, we all get a place for thoughtful discussion with reasonably fair moderation.
It's a fair trade for those involved and closer to a non-profit model than Reddit which is clearly trying to extract as much value as possible from the content creators on their platform in exchange for giving them community/platform.
How do you know? Just because there is no ads doesn't mean they don't have any benefits from keeping HN thriving.
Also for me I like HN because of the content, not because I care about YC's motive. You are obviously different.
Should we tell him?
Reddit could potentially do this but I don’t think they have a solid business model to support it yet. They are scrambling for money streams, not content. Although high end supported content drives ad views so who knows.
Some of Reddit’s success can be attributed to the difficulty of getting your personal blog indexed about your ‘87 Nissan Sentra anymore on Google.
Still, maybe Google also derives a lot of value from Reddit.
Would either of the two risk their position to extort some money from the other?
Discussion on official post here: https://news.ycombinator.com/item?id=39471317
In this instance "The Google" has expressed "an agenda".
I disagree that individuals might have been trying to wrong society but unfortunately it's the measurable results and direction of the company as a whole that exhibits the agenda and in this instance it's quite a blow to the ability for society to record a factual account of history so it can learn from its mistakes.
This seems to be the case for the latest tech. in general, suddenly upending the taxi industry, forcing small businesses to pay the gig-culture delivery tax, AI mass generating content and creating an imbalance in creative output.
We have a fun decade ahead...
The parasites at the top have $ in their eyes rn
I can't wait to search for who the Republican nominee is, and Google tell me to kill myself.
I’ve not seen this. What do you mean?
https://old.reddit.com/r/toronto/comments/100j6md/new_year_n...
https://old.reddit.com/r/toronto/comments/10t393y/final_upda...
https://old.reddit.com/r/toronto/comments/17i5x9u/posts_rela...
https://old.reddit.com/r/toronto/comments/17tszml/free_pales...
https://old.reddit.com/r/toronto/comments/18jctpk/propalesti...
https://old.reddit.com/r/toronto/comments/1apxrbc/torontos_m...
Not even allowing discussions of crime and city-wide threats is completely asinine and the mods reasoning is that everybody who visits r/toronto is racist and can't be trusted to discuss these issues in good faith. Yet .. it has NEVER been tried and this "experiment" has been going on 2 years now. What I think is really going on is that the Toronto mods are lazy and using that to implicitly pushing an agenda based on what they allow to be posted.
I'm not necessarily complaining that it's left leaning, it always has been in US parlance at least. My worry is that the tone as changed from disagreement to some form of disgust and hate, and yes even at times calls for murders. Maybe that's just the landscape now, broadly?
Just look at some of the 'mainstream' comments linked below. This is probably a 'both sides' thing, but I don't see the other side represented here and I would have equal concerns if it were in similar manner.
That said, I don't have a Reddit account, so just opened the site to see what it showed me. I'm not sure of its algo, as a lot of it is video games, but here are a few that showed up in the top 15 or so -
https://www.reddit.com/r/nottheonion/comments/1axe9rk/darryl...
https://www.reddit.com/r/LeopardsAteMyFace/comments/1axas1q/...
https://www.reddit.com/r/AnythingGoesNews/comments/1axad5x/t...
head over to /news, and you see -
https://www.reddit.com/r/news/comments/1ax8mnl/tax_evasion_b...
https://www.reddit.com/r/news/comments/1axe1ac/darryl_george...
/politics, you get
https://www.reddit.com/r/politics/comments/1axa6m9/donald_tr...
https://www.reddit.com/r/politics/comments/1ax49rq/alabamas_...
Yeah, about that whole "human discussion" thing...
And never use the internal search, I don’t know what gp is talking about. Reddit’s search is terrible.
These days at least 90% of product-related discussions on Reddit are astroturfed, though, and that's for sure.
The main problem, among many others, is that it's far too easy to game the upvote/downvote system with bots, and most pros have bot networks of their own, or know how to hire them. The second problem is that Reddit "karma" is site-wide, which means that most "high karma" accounts that are sold on secondary markets are very useful for digital marketing purposes generally. The third problem... well... I could go on all day. Reddit's system is horrendous if you're looking for human discussion and unbiased advice.
> You retain any ownership rights you have in Your Content, but you grant Reddit the following license to use that Content:
> When Your Content is created with or submitted to the Services, you grant us a worldwide, royalty-free, perpetual, irrevocable, non-exclusive, transferable, and sublicensable license to use, copy, modify, adapt, prepare derivative works of, distribute, store, perform, and display Your Content and any name, username, voice, or likeness provided in connection with Your Content in all media formats and channels now known or later developed anywhere in the world. This license includes the right for us to make Your Content available for syndication, broadcast, distribution, or publication by other companies, organizations, or individuals who partner with Reddit. You also agree that we may remove metadata associated with Your Content, and you irrevocably waive any claims and assertions of moral rights or attribution with respect to Your Content.
(https://www.redditinc.com/policies/user-agreement-february-1...)
tl;dr: "We will be able to do anything we fucking want with anything you contribute to our site"