Hacker News Reader

Comment by juujian | Hacker News Reader

My prediction is that pre-2022 data in machine learning will be like pre-WW2 steel in the medical industry (no radiation contamination from nuclear tests, cannot be replicated). An irreplaceable resource with a limited supply. As much as I hate to say it, Reddit may be sitting in an actual gold mine.

View on HN

That prediction and analogy to low-background steel has been made 128+ times before on HN: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

Never realized I was so predictable

Are you sure you aren't an ai already? That's also one reason I'm not so sure about the analogy because humans are already so gigantically redundant and we have lots of other bots already. I don't think there's ever been a low background steel on the internet in any real capacity. Just now it's more radioactive than it used to be

I will check next time I have an opportunity. Ironically, the only way to prove I'm not an AI would be for me to post content that would be restricted in language models, so here comes my proof: shit©

That proves nothing, you could just be any of the uncensored LLama or Falcon models.

>Never realized I was so predictable

"The limits of my language mean the limits of my world"

Our entire personality and sense of identity is nothing but the reflection of ourselves in relation to eachother. The summation of all cultural input. A man in a void is no more than a beast.

Here's Tom with the weather

"It's gon' rain."

Sideways!

That sums up about 99.9% of humans saying things.

So really there’s not a distinction between language models and people. Other than the lack of decision making and planning, the actual communication side is identical.

I think thats what's so unnerving about this stuff and driving all the "yeah but it actually sucks" type of articles.

The possibility that we really are just stochastic parrots is too much for people.

Language is a virus.

I've seen no evidence ChatGPT has decision making or planning capabilities.

Then check out autogpt?

>No decision making ability

What prompts are you using? It helps me make decisions every day involving complex and nuanced criteria. GPT-4 (especially 32k API + plugins) is better than 95% of the directors, PMs, and CEOs I have worked under.

Exactly, that’s the feature we have that they’re missing

Sort by date <https://hn.algolia.com/?query=low%20background%20steel&sort=...> and skim through and you’ll see it’s way lower. The considerable majority of those matches are about actual low-background steel for its own sake, not for analogy with pre-LLM data (or once, https://news.ycombinator.com/item?id=32097752, pre-microtransaction games!). Nuclear-related topics come up pretty regularly on HN.

Five pages of results; past the first page, there are only five relevant matches. https://news.ycombinator.com/item?id=23895706 three years ago (when it wasn’t yet a problem, but it was obvious it could become a problem), then the rest all within the last year (mostly this year) (when it is obviously becoming a problem), about thirty in total, half of which are multiple related comments. So the true number of times the prediction or analogy has been made is under twenty so far, and they’re mostly saying “like someone recently said” rather than coming up with the analogy by themselves.

The number seemed unrealistically high and I was curious. Now I have an answer! :-)

Thanks for clearing that up! It's nevertheless interesting to observe how multiple people come to the same analogy, or may be unconsciously affected by reading and then forgetting about it. Unfortunately I can't edit my original comment anymore.

> pre-WW2 steel in the medical industry (no radiation contamination from nuclear tests, cannot be replicated)

Interesting rabbit hole this one... but FWIW it's not as dramatic as this comment makes it sound. Newer steel is good enough because background radiation has dropped significantly since the 60s, plus there are non standard steel making techniques that don't lead to contamination.

Haven't gone down that rabbit hole for a while, thanks for the update!

Depending on the use case newer steel made in a standard fashion isn't always good enough. But indeed it's not that it cannot be replicated, but just that it's often more expensive to replicate than it is to just recycle old steel instead.

I wonder if the seeming uptick in cancers for people born in the 50s is related to just how many test explosions were done?

I cant pull up the stats via chatGPT, but I can pull up cancer stats from NIH - but the goal is to look at cancer death rates for the population of Las Vegas Nevada by each decade - as it states that radiation background has dropped since the 1960s -- It would be interesting noting births in Las Vegas starting in 1950, and comparing that with cancer rates btwn 1965-1975 compared with the number of nuke tests done in NV during the 1960s

My grandfather was a nuke eng for GE for 50+ years, he was on the design team of Hanford...

He died of thyroid/throat cancer via exenguination and we won a lawsuit against GE for nukes that were exposed to high radiation levels unbeknownst to them for exposure and risks/threats...

I might be wrong but I think they are sitting on the corpse of the golden goose.

Brilliant metaphor!

A dead golden goose containing one last giant golden egg. They don't care if it's dead, so long as they can cash out on the archived data that can be sold for training.

> corpse of the golden goose

a unicorn, one might say

Unfortunately for reddit, there are many backups of that data that aren't held by reddit.

Yep, a fuckton of it has been archived.

But does Reddit still control the legal rights? Who holds the copyright, and rights to use the data for various purposes?

If some new AI was trained on those backups, could Reddit take the creators to court for using that data without a license?

The users own their comments afaik, reddit only holds a license to them.

Are closed source products forced to reveal their training materials? Would be interesting to see a new company come out be closed source have a killer product then get forced to reveal training data in court only to reveal they pretty much only used stuff they didn't have rights to. Is it technically illegal yet to "learn" from data you don't have ownership of yet?

If somebody uses it and achieves some success, you can bet the truth will come out. Whether whistle blowers or crafty people on the internet piecing it together, people are pretty good at figuring that sort of stuff out.

Take that one step further - what about learning on generated data from a model that used copyrighted data? It is exactly what MS "Orca" does. Does this step launder copyright?

If you tried a language model pre 2017, you'd see it was not even able to produce two lines of coherent text. Now we are past coherence to the point we complain it bullshits in an authoritative style. That's of course because we use writing style as a shortcut for writing quality, not the best idea.

ahem OpenAI ahem

User-generated content belongs to the author. You need their permission if you want to use it. I don't know what the limit of a TOS is, I imagine misrepresentation for example is not allowed even if you write it inthere.

I've always thought it fascinating that people claim authorship or release things under a license without disclosing who they are. I imagine something licensed by Donald Duck is not actually usable under that license.

If it worked like that anyone could re-release anyone else's things under a different license. I mean, which Donald is the real author? How are you going to hunt the duck for violations? How do I prove I'm the real duck?

how would you prove what stuff was trained on? it's computationally infeasible to do so

What if, future AIs gain capabilities to sort, categorize, discriminate, deduplicate and rank new post-2022 datasets? When we ask for a text to be human produced, what we are really asking for is novelty and groundedness to facts. Most of the opinions on reddit is regurgitated hive mind thoughts.

Not the valuable stuff, the the world evolves and there’s an almost infinite supply of new stuff to talk about.

and yet here we are like a broken record talking about chatbots

A Comment section post, like language, is a virus.

What should we talk about then ?

Once you have a reliable model to detect AI-content, you also have a new tool to train AI-models against. So ironically, any tool to detect AI content will just serve to make AI content more difficult to detect.

Which you can use to train a better model to detect AI content.

I wonder whether some models fit this - well, you could certainly call it generative, and obviously adversarial - framework of networks?

The new AI models will seek novelty[0], pieces of facts not recorded before. It will directly hone in on newness not how it was delivered. Writing styles could be another vector in which novelty could be detected but that's a different matter.

[0] https://people.idsia.ch/~juergen/sice2009.pdf

But the ability to detect, say, content generated by pre-2033 chatbots will open up a whole decade of previously contaminated content.

right, you're describing a gan-like system/positional arms race. which is how adversarial tech always develops. doesn't mean we won't get better AI out of it.

I don't think that the ability to fuck something implies that there's an ability to unfuck something. I don't think there's any necessity to think that there will be an continually increasing ability to distinguish faked from real in past data.

The analogy would be how we can't distinguish fantasy, parable, and propaganda in histories written in the past, even though we have a lot more technology than they had 2500 years ago. We rule out most things that seem fantastic, like dragons or people rising from the dead (maybe the equivalent of finding extreme statistical improbabilities), while also entertaining the possibility that the past could have been radically different than we see it now, and some of the things we ruled out may have been real. We make suppositions based on the political alignments of the historians, and whether they could have had access to the information that they claim to record. We use geological and anthropological records to check credibility, and use confirmations found as sources for priors when judging other, maybe unrelated, things that said by that same scholar.

We've seen what current AI does when confronted by a situation like that, it fills in the gaps with plausible fantasy and hallucinates.

edit: not that there isn't low hanging fruit in detecting AI creations, but that's just a sign of early AI. AIs can't even draw hands.

Aye. Enshitification and Data Pollution appear to be thermodynamic processes. Easy to go one way, hard to go the other.

How about a new metaphor, complexification. The maximal complex state is useless and the initial simple state is also useless. Interesting patterns arise only in the middle like while mixing coffee and cream [0].

[0] https://www.youtube.com/watch?v=NgAtvbRqckQ

+1 on Sean Carroll. I think he overestimates the strength of human cognition, but that's not so vile sin as underestimating it. And he's a very good speaker I terms of getting his point across.

But yeah. There are definitely entropy analogies in software development. A couple of places I've worked at I've described as "brownian." Engineering would move forward a bit then marketing would change the requirements and we would make progress towards the new requirements, but before long we would get new requirements. If you mapped out progress starting at the origin and then put different objectives along the peremiter of a unit circle, progress would look like the random walk of an atom in a warm gas.

Those are the places where you ship a product when progress accidentally meets expectation.

A substandard programmer working within time constraints will produce bad results.

The goldmine has been archived and democratized the whole time: https://academictorrents.com/details/7c0645c94321311bb05bd87...

Reddit is a mithril mine and r/counting is the Balrog.

I had the realization this morning the people that really have a significant most are google because they have Gmail. Imagine how many people are sending messages every day via Gmail and how much of that text is human to human.

And if you believe that Google wouldn't use Gmail in a heart beat to start training their models once they figure out a good way to do it I've got a bridge to sell ya.

If you haven't read it, check out the story "Lena" about MMAcevedo: https://qntm.org/mmacevedo

Reddit has a lot of text on it, sure, but it's definitely not the best model out there, or should we expect language models that think it's appropriate to reply with "r/unexpectedoffice" for every single prompt?

I mean, Wikipedia is certainly better, but Reddit covers some niches of language that Wikipedia will never cover. Wikipedia, Reddit, and Twitter are the three predominant sources of data for large language models, and Twitter has been such a pain for researchers to access for a while already.

What about like, blogs, for instance, as a source of training data? Is the differentiating factor here something like, it's lots of people interacting with other people?

Blogs are hard. You have to manage a large collection of shallow links. People like reddit and Twitter because there are very few deep links. And in the old days you just opened up a pipe and Twitter streamed new tweets directly at you.

Pre-2022 Internet content definitely had spam/bot contamination though.

Of course they're sitting on a goldmine. If they weren't, they'd be absolutely mortified by what they're doing to their own website. I think by June 30th, it's going to prove to be a solid win for Reddit: they still have all of the data they ever had, and they'll have better ad revenue than ever. Unless the community can get a lot more creative, Reddit is about to be absolutely fucking rich (even though the communities will all be hallowed out.)

The average Joe enriching the elites by voluntary unpaid work. What a time to be alive.

It's kind of funny to me, though, that the mods just recently realized that's what they've been doing all along.

Certainly a time to be a landowner. For the rest of us, serfdom beckons even more inescapably.

It is sorta funny... I came to sili valley a moderate republican. I left it a marxist.

This is why anyone serious about "protesting" the changes should be deleting their account and overwriting/deleting all their comments and posts. Does it suck for the future, and for anyone trying to use that wealth of collective knowledge? Yes, of course. But that's also the only way to really hurt them at this point. Subreddit blackouts are completely ineffective, and drops in traffic with resultant dips in ad revenue are momentary blips. It's scorched earth or nothing.

Honestly, I think they're intentionally undeleting comments and un-editing them right now. Why wouldn't they? I'm pretty sure they legally can, and they clearly do not care that people hate them. I mean personally, I wouldn't feel safe if I worked for Reddit right now, but I am guessing the executives could absolutely not care less if anyone gets hurt over their amazing plan to get absolutely filthy rich.

"Reddit is about to be absolutely fucking rich (even though the communities will all be hallowed out.)"

If the communities will be gone for good (not likely at this point) - who will they be serving ads to?

The users producing most of the value that made Reddit interesting when it was a small website that had a lot to prove have probably already left. The thing is though, it's now very well-established. Reddit shows up extremely often when you Google it, and undoubtedly there's a lot of people (possibly a majority) who will just kind of shrug this off after deciding they'd rather suck it up than be without something like Reddit for a while.

But while the ad revenue is probably fine, the API revenue is definitely what they think the cash cow will be, and practically speaking they could lose 75% of their users and still probably not lose a whole lot of value just due to the sheer amount of historical data they have (especially if they start selling deleted content.)

There are signs that the blackout will have an affect on their bottomline if it's kept up [1]. I'd imagine that's why they're making these moves to remove moderators participating in the blackout so quickly. I think it'll be interesting to see if they can find other moderators who participate for free and bring as much value to reddit as the current mods do.

My guess is that the changes will still go ahead and the general quality of content on reddit will go down longer term although how much is an open question.

[1] https://www.adweek.com/social-marketing/ripples-through-redd...

If reddit doesn't do this, the bots will take over anyways. They probably already have.

AI is going to do a shitty job identifying AI, so all of your datasets are going to get worse and worse. I could imagine a bunch of generative AI bots just melting down every message board on the planet to the point where you have to stop training on recent data.

I was hoping for the bots to export their existential crisis' to the rest of the web. Bots producing a loop, where they train on the data that they produce, would create some insane results.

I believe you mean "hollowed out" and not "hallowed out." There's an important distinction between the two words.

Good catch, although now I'm outside of the edit window :)

Good enough just to verify your intent. I mean. It seemed obvious to me, but this subject matter seems to encourage people to wax poetic (and sometimes religious.)

If you search around a bit the data can still be found. I don't know about the legal part of this so I'm not going to post any links.

do you refer to Reddit data? Or Common Crawl before 2022?

Actual Reddit data from pushshift (compressed *.zst).

It's posted on academictorrents.

What about it is valuable? Isn't all of that "data" just their content that is pretty easily scrape-able by anyone?

Not only this, but also books.

There's a big difference between ghostwriting (biographies and fiction) and then the now fully GPT-generated books (non-fiction, especially technical) for sale on Amazon.

I'm very not sure about this

Text quality in reddit is awful. Especially comments . Forget using it for any semblance of actual grammatic correctness.

Probably can be used, after heavy cleaning and curation.

I think in this context, grammatical correctness is irrelevant, and what you actually want is just raw text as written by real humans from all walks of life.

Yup, variety is a big part of it. You don't just want a large language model that can produce meticulous text, you also need to be able to handle user input, which will frequently include faulty input itself. Otherwise you end up with something as mechanic and inflexible as google assist. Interactions with large language models can be much more organic if you have this kind of input data.

Agreed. We can make infinite grammatically correct text with perfect spelling, but if we limit the training set to that alone then ppl who wright liek this cant uze the ai lol & will be all lije "wtf m8 this aint skool i don care bout spekling rite u know wot i ment y cant ur machine?!?"

…

and while I enjoy my ivory tower as much as the rest of us, I do think it's important for the AI to understand everyone, not just me.

Depends on what you are looking to create. Good writing? Then it is an issue. Authentic seeming writing? Then it is exactly what you want.

One compensatory effect is that the noise will be more coherent than it used to be. Imagine how much entropy old SEO copy or spam could introduce to a model.

I think that as long as actual human beings continue to contribute content, AI would eventually "refine" that out and incorporate that into newer models. Even if say, 95% of the content in the dataset is AI generated, it will still capture the remainder.

Of course, I don't know if scaling AI with more data will work to improve it or not, and I'm sure that anyone really knows right now.

We can introduce healthy cows into the downer cow population. Even if say, 95% of the herd is exposed to BSE, it will still capture the healthy parts of cow carcasses we feed the downer cows.

(Which is to say... I think there are more than homeopathic concentrations of questionable training data. Or at least I believe that was the assertion of the OP.)

Maybe we will need “human farms” in response to bot farms. Maybe “professional human” will be a viable career option if human created data accrues extreme economic value.

> Reddit may be sitting in an actual gold mine

A pretty depressing thought to imagine our societies run by a bunch of ai redditors.

It's far more likely that training methods will improve. Humans don't have this issue.

are there any scraped copies of reddit data anywhere?

Reply on news.ycombinator.com