Comment by TeMPOraL | Hacker News Reader

Comment by TeMPOraL | Hacker News Reader

Well, so can a nontrivial number of people. It's Harry Potter we're talking about - it's up there with The Bible in popularity ranking.

I'm gonna bet that Llama 3.1 can recall a significant portion of Pride and Prejudice too.

With examples of this magnitude, it's normal and entirely expected this can happen - as it does with people[0] - the only thing this is really telling us is that the model doesn't understand its position in the society well enough to know to shut up; that obliging the request is going to land it, or its owners, into trouble.

In some way, it's actually perverted.

EDIT: it's even worse than that. What the research seems to be measuring is that the models recognize sentence-sized pieces of the book as likely continuations of an earlier sentence-sized piece. Not whether it'll reproduce that text when used straightforwardly - just whether there's an indication it recognizes the token patterns as likely.

By that standard, I bet there's over a billion people right now who could do that to 42% of first Harry Potter book. By that standard, I too memorized the Bible end-to-end, as had most people alive today, whether or not they're Christian; works this popular bleed through into common language usage patterns.

--

[0] - Even more so when you relax your criteria to accept occasional misspell or paraphrase - then each of us likely know someone who could piece together a chunk of HP book from memory.

I keep waiting for the day when software stops being compared to a human person (a being with agency, free will, consciousness, and human rights of its own) for the purposes of justifying IP law circumvention.

Yes, there is no problem when a person reads some book and recalls pieces[0] of it in a suitable context. How would that in any way address when certain people create and distribute commercial software, providing it that piece as input, to perform such recall on demand and at scale, laundering and/or devaluing copyright, is unclear.

Notably, the above is being done not just to a few high-profile authors, but to all of us no matter what we do (be it music, software, writing, visual art).

What’s even worse, is that imaginably they train (or would train) the models to specifically not output those things verbatim specifically to thwart attempts to detect the presence of said works in training dataset (which would naturally reveal the model and its output being a derivative work).

Perhaps one could find some way of justifying that (people justified all sorts of stuff throughout history), but let it be something better than “the model is assumed to be a thinking human when it comes to IP abuse but unthinking tool when it comes to using it for personal benefit”.

[0] Of course, if you find me a single person on this planet capable of recalling 42% of any Harry Potter book, I’d be very impressed if I ever believed it.

I keep waiting for the day when people realise that IP law has been used and abused and thanks to Disney extended out for many, many lifetimes and all manner of dirty tricks/hacks to keep the late stage capitalism profit engine going.

I 100% agree that if an LLM can entirely reproduce a book then that is copyright infringement, overfitting and generally a bad model. I also believe that in this case, HP (and other popular media) is overrepresented in the training data because of many fan sites/literal uploads of the book to the Internet (which the model was trained on). I believe that any & all human writing should be allowed to be used to train a model that behaves in the correct way so long as that writing is publicly available (ie on the Internet).

If I watch a TV show that someone uploaded to Youtube, am I committing a crime? Or is the uploader for distribution?

I also find it hilarious how many artists got their start by pirating photoshop.

Laws can have been used and abused and still be important. I know it’s hard to believe but the independent artists who were already struggling need IP laws to survive.

Otherwise Disney and the like can just come in, make copies or derivatives, and profit without paying those artists a penny.

Which everyone usually agrees (or used to) is not a fair outcome.

But somehow giant corporations not named Disney taking the same work in the same extractive mode in order to create an art-job-destroying machine is totally fine because Disney bad?

Maybe most people making this argument are also all for UBI and wealth redistribution on a massive scale, but they don’t seem to mention it much when trashing IP laws.

Abuse of IP does not mean the law is not relevant.

Don’t you find it funny that corporations with market caps the size of small countries first sued people for singing Happy Birthday in public, and now pretend that IP is suddenly not really a thing? Do you really want to defend their interests?

What abou defending our own interests?

I couldn't care less about Meta or OpenAI or other tech companies as entities. I'm happy to oppose them when they act against our interests. But in this, my interests align with theirs.

It's not like the alternative is any better. I'm much more in a philosophical disagreement with the pro-copyright side, for one, but the other thing is, it too is represented primarily by other large corporations, and of the two groups, the LLM side has a much more honest and useful business model.

First, I think they are exploiting the old conundrum where your personal interests may seem to align with theirs, but taken together our interests do not align with theirs. Sure, as it stands, an individual may enjoy personal gain—at least for the time being, while their prices are low as they operate at a loss to capture the market—but I believe this brief individual gain comes at the expense of society as a whole, and if people in more areas (not just music) made this an issue, and those megacorps put their infinite armies of lawyers to work on licensing (I recall somebody here once saying “do things that don’t scale” or something like that), everybody would benefit much more[0].

Second, copyright, or more precisely IP ownership, is not the same as the exploitation you might have in mind. Consider that copyleft—which gave us Linux, Blender, etc.—exists thanks to the ability to exercise and defend IP rights. When a stronger party takes that ability away from a weaker party, that is exploitation; maybe it is not a coincidence that Microsoft is at the forefront of this, as what they are doing is very much in line with a generalized form of EEE.

(Let’s be honest, homegrown LLMs will never reach the level of commercial models, and that’s where the money is headed; even GPUs aside, no individual has the ability to scrape the entire Web the way they do, even less so now once the ruthless interests of those corporations sent every website scrambling to defend themselves against what is predominantly bot traffic with layers upon layers of captcha never seen before. They realized that if they steal, they have to steal a lot and very quickly in order for it to work, and they’re certainly hoping to get away with it.)

More abstractly, being able to say that you have created something is pretty important to our willingness to create; I can’t see how inability to make this claim with a straight face (because anyone can, and already does as you have no doubt seen on this forum, reasonably claim that it may have not been your work at all) is good for society generally.

[0] Starting with awareness of what is going on and the freedom to choose. Information asymmetry is the enemy of free market.

I most certainly would like to see tax havens abolished worldwide. And I would most certainly like to see these corporations and their executive pay the taxes that they should be paying.

But I've come to realise that the apathetic general public only care about being racist, sexist and homophobic to each other whilst the ruling class laugh all the way to the bank. And the few intelligent people who understand what's going on don't raise their voices so long as their high tech salary is protected, so long as their taxes aren't raised, so long as they can own a holiday home or two when others struggle for their first home.

If you actually look into the numbers, what tax companies like Apple, Google, Amazon etc actually pay it's just...yeah.

> I keep waiting for the day when software stops being compared to a human person (a being with agency, free will, consciousness, and human rights of its own) for the purposes of justifying IP law circumvention.

I mean, "agency" is a goal of some AI; "free will" is incoherent*; the word "consciousness" has about 40 different definitions, some of which are so broad they include thermostats and others so narrow that it's provably impossible for anything (including humans) to have it; and "human rights" are a purely legal concept.

> What’s even worse, is that imaginably they train (or would train) the models to specifically not output those things verbatim specifically to thwart attempts to detect the presence of said works in training dataset (which would naturally reveal the model and its output being a derivative work).

Some of the makers certainly do as you say; but also, the more verbatim quotations a model can produce, the more computational effort that model needs to spend to get the far more useful general purpose results.

* I'm not a fan of Aleister Crowley, but I think he was right to say that there's only one thing you can actually do that's truly your own will and not merely you allowing others to influence you: https://en.wikipedia.org/wiki/True_Will

> and "human rights" are a purely legal concept.

Yep, and if you claim that a thing can reproduce IP like a human then you should explain why you are also not holding its operators to the same legal standard (try to use a human in the same way and it will be considered torture and slavery).

I am specifically not using that to claim "and therefore the AI is a human". The point is that "human rights" are not part of the natural order, they only exist as laws.

This means that "human rights" is basically irrelevant to this topic: they may have rights and need to be liberated, or they may be tools that don't, but the law is just words on paper, and officials who make you follow those words.

> The point is that "human rights" are not part of the natural order, they only exist as laws.

I have seen this argument used before; when something suits an argument, it’s “nature”, when something doesn’t then it isn’t. I think it’s a fallacy.

Humans are part of natural order. Our laws and how they evolve are part of our nature, and by extension part of nature. I don’t believe in silencing such discussions as irrelevant because of an imaginary cutoff point where it stops being part of nature and suddenly becomes “artificial”.

> This means that "human rights" is basically irrelevant to this topic: they may have rights and need to be liberated, or they may be tools that don't, but the law is just words on paper, and officials who make you follow those words.

I’m not sure how it is irrelevant. If we can claim “LLMs are like human, and so their creators and commercial operators are not guilty of IP laundering that LLMs do”, then we have a moral imperative to stop using them because, well, they are like human, and no human should be put through what would be, to any human mind, abuse. If we do not believe they are human and free in this sense, then the excuse quoted above in this paragraph also stops applying.

By that tautology, so are rocks. Rocks don't get natural rights.

Also, you may note that "human rights" is a recent invention and not actually enforced worldwide even today.

> you may note that "human rights" is a recent invention and not actually enforced worldwide even today.

Consider that countries known for stronger interpretation of human rights and freedoms, including intellectual property rights, are also the countries at the forefront of innovation, including technical innovation that laid the foundation for LLMs in the first place. I think that is not a coincidence, and we should keep it in mind when there is a push to be dismissive of these concepts (which predominantly serves the interests of commercial LLM operators and their supply chain).

I’m sure you would not argue from a point where this recent interpretation of human rights is bad or incorrect, but if you would then perhaps there’s not much of a constructive discussion to be had. I would still oppose the use of the natural vs. unnatural distinction as the basis of that argument, though.

> Consider that countries known for stronger interpretation of human rights and freedoms, including intellectual property rights, are also the countries at the forefront of innovation, including technical innovation that laid the foundation for LLMs in the first place. I think that is not a coincidence, and we should keep it in mind when there is a push to be dismissive of these concepts (which predominantly serves the interests of commercial LLM operators and their supply chain).

Several fallacies there.

First because China and the USA are opposite ends of the spectrum on many of the ways "freedom" is measured, and yet China is doing pretty well on the innovation front, including with AI. And that China is beating Europe, even though various Scandinavian nations rank higher on such freedoms than does the USA.

Second, cum hoc ergo propter hoc: Correlation does not imply causation. For example in this case, a reason why one of the big IP groups in the USA (Hollywood) got big, was because being in California enabled them to avoid the IP rights of the Motion Picture Patents Company that dominated cinema in the East Coast. I would even suggest that it is the disregarding of IP rights that enables much of the web, not only how and why China is doing well, but also Google (which has had legal fights over the interaction between copyright and search results), social media, and cultural elements such as memes and reaction gifs.

Third: the point of copyright is to encourage new works, because this makes money which can be taxed. All this becomes somewhat irrelevant when AI can also create new works.

If you want to set a bar for creativity high enough that current AI can't reach it, I suspect quite a lot of human works also fail, e.g. that Pratchett's Strata is obviously Ringworld, and that you would exclude from copyright all parts of The Lion King that are based on Hamlet.

> I would still oppose the use of the natural vs. unnatural distinction as the basis of that argument, though.

I'm not sure what you're saying when you "oppose" this. Does that mean you accept that, in principle, there could be some AI which would deserve rights in the category currently (but in principle inaccurately) called "human rights"?

> China is doing pretty well on the innovation front, including with AI.

From transistors to transformers, most of it builds on foundation that comes guess from where. The innovative layer you speak of is fairly thin.

> Correlation does not imply causation.

I give you that. However, without being able to re-run history, correlation is all we have.

> I would even suggest that it is the disregarding of IP rights that enables much of the web

I would suggest that much of the tech that powers the Web, including probably the most popular operating system on which most servers run, is enabled by copyleft, and copyleft cannot exist without the ability to defend it granted by IP rights, the very concept under fire.

> the point of copyright is to encourage new works

I agree on this.

> All this becomes somewhat irrelevant when AI can also create new works

I don’t agree with a phrase “AI can create new works” for reasons such as 1) “AI” is a meaningless term (let it be my revenge for consciousness) or 2) a tool without agency or will should not be X in a sentence “X can Y” (sure, we can maybe on occasion say “hammers can break things”, but if hammers having agency and will was a popular misconception then I would definitely prefer to stick to “hammers can be used to break things”). The “create new works” part is also questionable on a few levels, but it might exceed the scope of this argument.

That aside, I believe lack of copyright enforcement discourages the creation of new works even in presence of these tools, through the mechanism known as “why would I put effort into new work if I don’t effectively own the result”.

> Does that mean you accept that, in principle, there could be some AI which would deserve rights in the category currently (but in principle inaccurately) called "human rights"?

I think if we believe an LLM or some other software is sufficiently close to a human that it deserves human-like rights or just strong abuse protections (cf. octopus in some countries)—without saying whether I believe it possible or not, it really is orthogonal—then we could excuse it reciting some part of Harry Potter in the right context (probably not as work for hire), but it would be moot because we would also be ethically compelled to not subject it to the training and use that enables such recitation in the first place.

> From transistors to transformers, most of it builds on the foundation that comes guess from where. The innovative layer you speak of is fairly thin.

China rises to first place in most cited papers: https://www.science.org/content/article/china-rises-first-pl...

> 1) “AI” is a meaningless term (let it be my revenge for consciousness)

Fair.

But let's say "computer program" in that case. I'm not fussed about definitions.

> 2) a tool without agency or will should not be X in a sentence “X can Y”. Sure, we can maybe on occasion say “hammers can break things”, but if hammers having agency and will was a popular misconception then I would definitely prefer to stick to “hammers can be used to break things”.

Careful.

If you say that humans can only create things with copyright (even if to support copyleft), then the proletariat are the tool that the bourgeois use to create things.

I do not think this is what you intended :P

> That aside, I believe lack of copyright enforcement discourages the creation of new works even in presence of these tools, through the mechanism known as “why would I put effort into new work if I don’t effectively own the result”.

Same reason you commission a work, or even just buy it from a shop: because then you have the thing.

I mean, the cost of getting o3 to create a novel worth of text is about the same as the price of a generic book by an unknown author in a second-hand shop: https://openai.com/api/pricing/

I've not tried o3 yet, but I have tried o1, and as I've said on a different thread today, o1's output is merely OK, not worth publishing as a book — and I don't know how long it will take to get there. But it is displacing blog writers and podcast writers: https://news.ycombinator.com/item?id=44287953

> but it would be moot because we would also be ethically compelled to not subject it to the training and use that enables such recitation in the first place.

Surprising.

While I would seriously consider the possibility it may be unethical to force such an AI to work if it didn't want to, I think giving it the capability, the education, to be capable of making that choice rather than just saying "it doesn't matter if I wanted to or not, I can't", is just education, as per our own.

Still, I think that's coherent. I'm not sure I've fully internalised the implications so I will let it be.

> https://www.science.org/content/article/china-rises-first-pl...

Per capita?

> If you say that humans can only create things with copyright (even if to support copyleft), then the proletariat are the tool that the bourgeois use to create things.

The wealth gap and the divide is unlikely to be helped if more people are going to be using (and paying for, in whatever way) ML-based tech from a handful of large corporations.

> Same reason you commission a work, or even just buy it from a shop: because then you have the thing.

Simple posession is more about physical necessities. Commissioning or buying artwork from someone is not just about posessing it, it comes with supporting someone financially. I could make icons or basic illustrations for some small project myself, but I would still commission them if I can afford it because that supports an artist who may want some work (as well as building up for more collaborations in future). Here, I would be supporting the opposite of those artists, a thing that was built on those artists’ work without their consent. Some middleman megacorp of the worst kind.

> the cost of getting o3 to

Don’t they operate at a loss for the time being? They will have to make money sooner or later.

> While I would seriously consider the possibility it may be unethical to force such an AI to work if it didn't want to, I think giving it the capability, the education, to be capable of making that choice rather than just saying "it doesn't matter if I wanted to or not, I can't", is just education, as per our own.

This goes way beyond my thought. I assume if we are talking about education it would be a given that running it generating images 24/7 non stop, shutting it down/killing it, etc., is already out of the question.

Agree completely. When I read the Gemma 3 paper (https://arxiv.org/html/2503.19786v1) and saw an entire section dedicated to measuring and reducing the memorization rate I was annoyed. How does this benefit end users at all?

I want the language model I'm using to have knowledge of cultural artifacts. Gemma 3 27B was useless at a question related to grouping Berserk characters by potential baldurs gate 3 classes; Claude did fine. The methods used to reduce memorisation rate probably also deteriorate performance in some other ways that don't show up on benchmarks.

> When I read the Gemma 3 paper (https://arxiv.org/html/2503.19786v1) and saw an entire section dedicated to measuring and reducing the memorization rate I was annoyed. How does this benefit end users at all?

It benefits users because memorisation is a waste of parameters that would be more useful if they were instead learning rules and generalisations.

For short snippets, common idioms and quotations that people recognise, exact quotes can be worth memorising; but the longer the quotations get, the less often it is important to be word-for-word exact — even for just a few paragraphs, I think most people only ever do oaths, anthems, songs they really like, and possibly a few hobbies.

If you want an exact quote, use (or tell the AI to use) a search engine.

Reply on news.ycombinator.com