Benn Jordan's AI poison pill and the weird world of adversarial noise
cdm.link
cdm.link
The answer is, almost unfailingly, "this paper applies perfectly to our technique because we are just rehashing the same ideas on new modalities". If you believe it's unethical for GenAI models to train on people's music, isn't is also unethical to trick those people into posting their music online with a fake "defense" that won't actually protect them?
In some of the earlier image protection articles the people involved seemed rather shady about the capabilities. Would have to do some HN searching for those articles.
But everything at the end of the day will be a scheme if the end result is for humans to listen to it. You cannot make a subset of music that can be heard by humans (and actually sounds good) that cannot be prefiltered to be learned by AI. I've said the same thing about images, the same thing will be true about audio, movies, actions in real leave, et al.
These schemes will likely work for a few of the existing models, then fall apart quickly the moment a new model arrives. What is worse for defense is audio quality for humans is remaining the same while GPU speeds and algorithms increase in speeds over time meaning the time until a model beats the new defense will trend to unity.
While the artist in question can have the best intentions, conmen will swoop down on this and productize it, and then artists will be sad and confused when it has zero long term effect on machine learning. That is, except making machine learning more resilient to adversarial attacks.
More specifically, there are a few moving parts here - the GenAI model they're trying to defeat, the defense applied to data items, and the data cleaning process that a GenAI company may use to remove the defense. So we can look at each and see if there's any reason to expect things to turn out differently than they did in the image domain. The GenAI models follow the same type of training, and while they of course have slightly different architectures to ingest audio instead of images, they still use the same basic operations. The defenses are exactly the same - find small perturbations that are undetectable to humans but produce a large change in model behavior. The cleaning processes are not particularly image-specific, and translate very naturally to audio. It's stuff like "add some noise and then run denoising".
Given all of this, it would be very surprising if the dynamics turned out to be fundamentally different just because we moved from images to audio, and the onus should be on the defense developers to justify why we should expect that to be the case.
What artists don't realize by this they are just improving the models relative to human capabilities. The adversarial techniques like, for example making a stop sign look like something else, well likely be weeded out of the model by a convergence of model performance to average or above average human performance.
But, since LLM is so error prone & AI companies don't seem to want to pay humans to verify either the data being input into LLM training is valid, or the output is accurate, something like a forced CAPTCHA to be used for verifying correct LLM data by unpaid labor.
It's just a dystopian thought I had. I probably shouldn't have said it outloud (it might give them ideas).
[1] https://www.techradar.com/pro/security/a-tracking-cookie-far...
For early experiments people literally took Stable Diffusion and fine tuned it on labelled spectrograms of music snippets, then used the fine tuned model to generate new images of spectrograms guided by text, and then took those images and turned them back into audio via re-synthesis of that spectral image to a .wav.
Riffusion was one of the first to experiment with this, 2 years ago now: https://github.com/riffusion/riffusion-hobby
The more advanced music generators out now I believe have more of a 'stems' approach and a larger processing pipeline to increase fidelity and add tracking vocal capability but the underlying idea is the same.
Any adversarial attack to hide information in the spectrograph to fool the model into categorizing the track as something it is not isn't different than the image adversarial attacks which have been found to have ways to be mitigated.
Various forms of filtering for inaudible spectral information coupled with methods that destroy and re-synthesize/randomize phase information would likely break this poisoning attack.
TLDR: Once these defenses are broken, all previously protected work is perpetually unprotected, so they are flawed at a foundational level.
Ignoring these arguments and pretending they don’t exist is pretty unethical.
But 'protection' of any one song isn't the entire point. It only takes less than a fraction of a percent of corpus data to have persistent long term effects in the final model, or increase costs and review requirements to those stealing their content.
As most training is unsupervised, because the cost and limited access to quality, human labeled data, it wouldn't take much if even some obscure, limited market, older genres which still have active fan bases, like Noise rock to start filtering into recommendation engines and impact user satisfaction.
Most of the speech protections, just force attacks to be in the perceptible audio range, with lo-fi portions like those of TripHop, that would be non-detectable without the false positive rate going way up. With bands like Arab On Radar, Shellac, or The Oxes, it wouldn't be detectable.
But it is also like WAFs/AV software/IDS. The fact that it can't help with future threats today is immaterial. Any win of these leaches has some value.
Obviously any company intentionally applying even the methods in your linked paper to harvest protected images would be showing willful intent to circumvent copyright protections and I am guessing most companies will just toss any file that it thinks has active protections just because how sensitive training is.
Most musicians also know that copyright only protects the rich.
https://securitycryptographywhatever.com/2025/01/28/cryptana...
It will be really interesting as this knowledge percolates into more and more fields, what domain experts do with it. I see ML as more of a bag of tricks that can be applied to many fields.
It's his art and his livelihood too, so it's also personal. These people want to steal his art and create a world full of soulless cheap muzak, while simultaneously putting him out of work.
Get 'em, Benn! I should go buy one of his albums.
I am curious if anyone read Harry Potter in bootleg form from a LLM. I mean, LLMs are the worst tools for infringing - they are approximate, expensive and slow, while copying is instant, perfect and free. You can apply the same logic for other modalities.
Moreover, who's got the time to see someone else's AI shit when they can generate their own, perfectly customized to their liking? I personally generated a song about my cat and kid. It had zero commercial value but was fun for 2-3 people to listen.
I can steal an invention without wiping the inventor's memory.
These are other kinds of stealing, which deprives the creator not of the art itself, but the other benefits of having created it.
Let's say we change copyright to cover abstractions, then any human creator would be in deep trouble. Incidental similarities would make the creative act too risky.
And are you not fearful of a world in which nobody ever develops a new Ghibli-tier art style, because they'd starve in the process while competing against low-cost AI interpolations of prior art and AI companies make a fortune selling their efforts?
I'm fearful of a world in which anyone can print "Huggies" on counterfeit diapers and pass them off on Amazon as the real deal, so I hope you at least have room in your philosophy for trademarks, if not trademark art styles.
But either way, the good news is that you yourself can put in the work to hand-paint 100,000 frames of imitation Ghibli-style artwork and then train an AI on your portfolio and give it to whoever you want, or even master a style of your own; the bad news is that this will take sixty years of your life and career, during which time you'll have to feed yourself and your staff, and when you're done you might not feel so generous about giving it away for other people to use to sell greeting cards on Etsy.
But there's more good news, which is that Miyazaki is at least generous enough to allow at least one other studio to use his art style with his blessing, as long as the result lives up to the following standard: "every film you make, you’ll have to realize that has to be a film that is worthy to show to children".[2]
And personally, I do think he's earned the right to ask that much.
[1] https://theaiunderwriter.substack.com/p/an-image-of-an-arche...
Anyway, focusing on training data is misguided unless you also restrict in context learning by policing what users can paste into the model.
It's entirely valid to consider the act of training via ML a right granted by the creator like reproduction or performing, simply on the basis of protecting human art. The comparison with human learning can be made irrelevant (and IMO it's not the same fundamentally).
This is currently a discussion about plagiarism (the ethics) and what the outcomes are from unrestricted GenAI. How copyright applies to GenAI is a question for later, informed by the discussion by society at large (and lobbyists).
No, it isn't. It is a widely understood if not explicitly stated and frankly, so easy to understand concept that I'm forced to conclude people as yourself who refuse point blank to understand it are merely doing so out of sheer determination.
When you post things online, be it art, be it music, be it photographs, be it the idle thoughts of your mind: you do this to share it with other people. That's the point. Why you want to share it can certainly change and explanations range from desiring an online following/clout to making money to simply the fact that you are driven to create and share whatever it is you do. This is the entire basis of the proto-Internet, to a degree where it predates the whole notion of extracting wealth from others via the Internet, and to a large degree, also predates the notion of a following.
If you desire to counteract this premise, then I would suggest you must somehow explain the existence of BBSes, newsgroups, anonymous web forums, blogs, etc. that are some of the oldest, dustiest pillars of the Internet, that our modern Internet would not exist without, and moreover, tons of these existed when analytics, statistics, etc. were barely conceived of. Some websites had a guest book, and some had visitor counters, and that was literally it. If you published on a blog like I did in the 90's, you had no fucking clue if anyone at all was actually reading it.
Or to put it shortly: The Internet and it's constituent communities existed long before there was a financial motive to publish, and absent that, what possible explanation could there be to publish apart from have your thing seen by people?
Ergo, to then take those intellectual products, shared in good faith for the enjoyment of others, run them through an industrial shredder, identify the most common patterns in their underlying structure, and sell it back to the public to whom it's existence simply must be credited, is perverse. It is self-expression without the self; it is merely expression, for the sake of expression. Noise. And we all kind of know this, prior to LLMs, the most common experience of this phenomenon was spam. Because if you simply posted links to whatever thing you were hocking it would be deleted immediately, spammers had to get clever. They would write what were ostensibly "blog posts" but they were not sincere expressions of anything apart from avaricious greed and a desire to camouflage it. Websites rose that were nothing but endless pages of this shit, designed to be optimally crawled by Google and other engines, linking back to sites seeking favor in search engines. Web forums at that time had to regularly moderate away content not for being hateful, but for being openly fucking stupid, irrelevant to topic, and simply seeking to drive traffic to other sites. Anywhere there was an unprotected text input who's contents may, eventually, arrive at a page was, to a spammer, a perfectly valid place to park their shit that nobody wanted, and park it they did.
And this is exactly what LLMs make. Soulless nothing who's primary reason to exist is to occupy space. Image generators are the same; their outputs exist because an image, no matter how generic or mediocre, would look better in this spot than a blank color. Music, without motivation or drive behind it, as a convenient alternative to silence. Video because an image would be boring. Functional, purposeless, driveless, motivation-free content to occupy the otherwise empty void around whatever thing you're actually trying to get eyes on: Text to cloak your links, mediocre art to cloak your sales pitch, etc.
And the reason the business parasite class is so thrilled about it's existence is that it finally allows people who have not nurtured creativity within themselves for whatever reason you'd care to assign access to creative outputs. They have sneered derisively at people who feel the drive to create and learn the ability to do so because fundamentally they do not believe creative products have value, that the attention, the adoration, and more than anything the money that said creatives are given so freely by their fans would be far better in their pocket instead. You can hear this in the way they talk: "I'm making a movie with AI!" said as though it's a revolutionary statement, as though great directors announce "I've started production on my next picture using Red Digital Cameras" like anyone from the public would give an ounce of a shit how a thing was made. They don't even consider "will someone want to see this movie?" because other people seeing it isn't the point, their elevation is the point. The promotion of their brand is the point. They have nothing at all to say, and it will doom anything they make to abject failure.
AI permits capital to access skill without requiring at all that skill be permitted to access capital.
Also information wants to be sorted and compressed. See for example personal wikis using org-roam, personal databases of text files using org-mode in emacs and so on.
What good is for me, 10 songs which each one has 10 seconds of a part i like, and i am indifferent to the rest of the song. I take these 100 seconds in total and compress them down to a new song which is totally to my liking.
Categorically, no. When everyone has money, in fact, no one has money. If you don't believe me ask someone from Zimbabwe.
> When food, medicine and narcotics become information with genetic engineering, then everyone has food, medicine and narcotics.
The barrier to everyone having food, medicine and narcotics is not money, it's the restriction of those important goods to people who have money; because food is not grown because people are hungry, nor are medicines made because they are sick; both of these things are done for profit, and people without money are unprofitable to the producers and distributors of those goods.
We have enough food for everyone almost twice over. The problem is the majority of it is pumped to overweight westerners who have disposable income.
> What good is for me, 10 songs which each one has 10 seconds of a part i like, and i am indifferent to the rest of the song. I take these 100 seconds in total and compress them down to a new song which is totally to my liking.
Oh that's so nice for you. And all it took was the bankrupting of every music artist.
Your views on this are so myopic as I genuinely wonder if you're a parody account. I didn't think it was possible for someone to be this short-sighted and selfish but here we are.
Let's say someone in Zimbabwe wants to buy olive oil. He has to buy from the top producers in the world, i.e. Spain, Italy and Greece. I will sell olive oil to Zimbabweans only for profit, if they don't have money they will not have olive oil.
But let's say a smart Zimbabwean figures out a way to genetically modify Chlorella or Spiroulina which only need sun and no particular climate or fertile land and he produces so much olive oil that it's price collapses to nothing. Not only that will bankrupt me, but such a new innovation in food production will put me at a disadvantage of even competing using the new innovation because the reason is simple, i don't have even half the sun Zimbabwe has.
But even if my income stream of olive oil goes to zero, i can sell something else, i can even learn to program computer code, and sell that. Or learn genetic engineering myself and create some new innovation. There is never any shortage of problems to solve, and things to produce.
That's a hell of an industrious Zimbabwean, firstly.
Secondly though, while I understand where your metaphor is aiming, it just doesn't really work. There has never been a "business with a competitive advantage" on the scale of human creative output vs. AI. I would argue even this example you've cooked up which already strains belief on several points doesn't truly capture it.
AI in your example wouldn't be one person in one place that can make a product at unbelievable scale; I think it would be more analogous to everyone on Earth suddenly having a machine on their countertop that produces Olive Oil by itself, of low quality, for free, forever. And I think the effects would be similar: any producer of less than fantastic olive oil would go bankrupt, because they're effectively competing with a free product (not unlike the Netscape Navigator vs Internet Explorer problem) and most consumers who are fine with a low-grade oil have zero reason to buy more when they can just buy access to the infinite olive oil maker. You would also then have producers of food both industrial and restaurants who would begin advertising that they use real proper olive oil, many of which would doubtlessly be using the shitty stuff from our imaginary machine or perhaps a blend of both to cut costs, but nevertheless, this hurts the producers of the good stuff too, because again, they are competing with a free-at-point-of-use product.
And like, on balance, all of this is just a net negative for everyone:
* A lot of the olive oil is worse now, for everyone, seemingly permanently absent regulations
* People are now losing a lot of counter space to the machine
* We've obliterated an entire agricultural sector and farmers who have done nothing but this their entire lives have to retrain jobs and/or their land has to be used to grow something else
* One company that made the infinite olive oil machine now has billions and billions of dollars, that they have earned by making the other three things happen
And just... why? Why is this a good thing for humanity? Put aside the stupid game with the made up money and line go up and justify to me why we've now done this, because it seems like everyone has lost a little bit, some have lost a lot, and a vanishingly tiny minority have won big, explicitly at the expense of everyone else.
And before you start with "well the market chose-" no, it didn't. The market followed incentives established by the market and those who regulate it. That's not a choice anyone or anything made, any more than it's a choice if a river floods it's banks and destroys a bridge.
Any musician these days that thinks there is money in music by selling songs is delusional. Sad but true.
The Flashbulb - Parkways: https://youtu.be/C6pzg7I61FI
There's new models showing up regularly. Civitai recognizes 33 image models at this point, and audio will also see multiple developments. Any successful attack on a model isn't guaranteed to apply to another one, not even yet invented. There's also a multitude of possible pre-processing methods and their combinations for any piece of media.
There's also the difficulty of attacking a system that's not well documented. Not every model out there is open source and available for deep analysis.
And it's hard to attack something that doesn't yet exist, which means countermeasures will come up only after a model was already successfully created. This is I'm sure of some academic interest, but the practical benefits seem approximately none.
Since information is trivially stored, anyone having any trouble could just download the file today and sit on it for a year or two not doing anything at all, just waiting for a new model to show up.
Seems like an awful risk to deliberately strip such markings. It's a kind of DRM, and breaking DRM is illegal in many countries.
For instance, I've seen somebody experiment with Glaze (the image AI version of this). Glaze at high levels produces visible artifacts (see middle image: https://pbs.twimg.com/media/FrbJ9ZTacAAWQQn.jpg:large ).
It seems some models ignore it and produce mostly clean images on the output (looking like the last image), while others just interpret is as a texture, the character is just wearing a funny patterned shirt. This is while the intended result is fooling the model to generate something other than the intended character.
This sounds like you’re talking about img2img generation based on a glazed image instead of training, which isn’t the intended purpose.
Here's an example I found: https://www.reddit.com/r/aiwars/comments/1h1x4e2/a_very_deta...
You can see there an example of training picking up the glaze artifacts and just using them as a funky texture. That's not really what Glaze is intended to do. Glaze is supposed to interfere with training, not be interpreted as "this artist draws skin with a weird pattern on it".
Because of that reality, every artist who wants to make money must either participate in it, or completely isolate themselves from it.
These models have become an incredible opportunity for giant corporations to circumvent the law. By training a model on a copyrighted work, you can launder that work into your own new work, and make money from it without sharing that money with the original artists. Obviously, this is an incredibly immoral end to copyright as we know it.
So what are we going to do about this situation? Are we really going to keep pretending that copyright can work? It wasn't even working before all the AI hype! Ever heard the words "starving artist"? Of course you have!
We need a better system than copyright. I'm convinced that no system at all (anarchy) would be a superior option at this point. If not now, then when?
I do wish, though, that he would have introduced that perspective of the situation in this particular video. Leaving it out feels like making a video about learning to swim, set in the middle of the ocean.
Not sure if "you" refers to model developers, hosting company or end users. But let's see each one of them in turn
- model development is a cost center, there is no profit yet
- model deployment brings little profit, they make cents per million tokens
- applying the model to your own needs - that is where the benefit goes.
So my theory is that benefits follow the problem, it is in the application layer. Have a need, you can benefit from AI, don't need it, no benefit. Like Linux. You got to use it for something. And that usage, that problem - is personal. You can't sell your problems, they remain yours. It is hard to quantify how people benefit from AI, it could be for fun, for learning, professional use, or for therapy.
Most gen-AI usage is seen by one person exactly once. Think about that. It's not commercial, it's more like augmented imagination. Who's gonna pay for AI generated stuff when it is so easy to make your own.
When someone creates art, copyright says that there is a countable result we can refer to as their "work". Copyright also says that that artist has a monopoly over the distribution and sale of that work. The implication is that the way for an artist to get paid for their labor is for them to leverage the monopoly they have been granted, and negotiate a distribution scheme that involves paying them.
When an artist chooses to work outside the copyright model, that means they must predetermine part of their distribution negotiation. That might be the libertarian option (gratis distribution with no demands), or it might be the copyleft option, where the price is demanded, but also set to 0. The artist may find payment for their labor by other means, but that's challenging to do in an economy where copyright participants dominate.
the [transferability] rates just drop off significantly for audio (always felt it was a similar vibe to RNN ‘vanishing gradients’)
edit — specifically mention transferability
I hope that the adversarial attacks can be easily detected and circumvented, just like other IP protection measures have been subverted successfully.
It doesn't really seem like an IP thing to me - it's like making music that's intentionally hard to classify in a particular genre on purpose or something. Maybe that defeats Spotifys recommendation algorithm but that's totally unrelated to any rights to stream it or so on.
Eventually encoders/classifiers will get an 'anthropomorphic' loop of human ear capabilities to use as a prefilter and identify when sounds differ from the expected human hearing and identify the noise.
I mean this is what adversarial reinforcement learning is, humans cannot beat AI on this in the end.
What's particularly funny is when that extremist politician is too stupid to understand the meaning of the song they're playing. I think the most famous example would be a certain somebody who kept playing Fortunate Son at his rallies, both him and the crowd oblivious to the fact that nearly every verse's criticism directly applies to him. A rich kid who got out of serving in Vietnam, cheated on his taxes at every opportunity, and is screaming about how great the country is while vilifying a demographic that has always been one of the strengths of our society.
I'd say that "respecting the property rights of the creative class" is very much indeed a 'left wing' perspective, too. Not just on a moral issue of "stealing their work is wrong" but also because artists very often mock or satirize right-wing personalities and platforms - punching up. Right wing artists tend to just punch down at society's weakest.
(This obviously doesn't cover every single person -- some are just legitimately pro-IP.)
There's a big corollary to FOSS and gpl, but a lot of people seem to miss it. This future isn't for our culture, it's for business. And since our culture requires making money to make things adjacent to art, less culture will be made.
A lot of generated music is good enough to wind up in playlists. Lots of live bands already fake it, and lots of music venues have already replaced live music. That doesn't mean music will die, it just means it's harder to discover new music and go see a show.
I think there's some European countries that have some kind of basic income/stipend for artists. If this is the direction of the future we need to ensure something like this exists.
This type of vague posting is very popular in HN.
> X thing was originally universally leftist (it wasn't), now it's right wing only (it isn't), something something the real counterculture or grey tribe.
Leaning from music doesn’t mean imitating that music. People can for example learn from mistakes without imitating those mistakes.
Is that your stance? Or do you have some more limited rejection of IP in mind.
Myself I don't see those as being related at all. Keeping something secret is allowed. What is not allowed is something being 'not-secret' yet special. Once you share the source code to the world there would be no closing the barn door.
What and how you can keep a secret is very much under the term IP, if you want to describe something more specific don’t use terms meant to describe something else. If you’re full on “information wants to be free” then the line doesn’t stop at a companies firewall.
Intellectual property is about someone having ownership over an idea. It does not mean that you have a right someone gives his idea to you. These are totally separate things.
I am free to write, paint, code or film whatever I want. I have zero obligation to share that with anyone.
Remove it and you can’t retaliate against anyone buying or selling access to your company’s source code thus making it impossible to keep private. So all such source code ends up public at any reasonably large company effectively killing all AI startups etc. The same is true of any proprietary information thus quickly killing nvidia, AMD, etc. Medical R&D also dies outside of governments and non profits. It’s shocking just how many industries get gutted as there’s far less incentive to make accurate maps when anyone can copy them.
But hey, I respect your stance here.
PS: If you think contracts/DRM/whatever that’s the solution! Now you do have protection for info and thus IP again.
I hate to give a smarmy result, but are you sure you know what RLHF is? Because this is one way to correct said data.
There’s a great deal of lesions to be learned from X PB of training data that wouldn’t be covered.
On the off chance that you are not: IP-rights does not cover "learning from" a source. What ML does is not in any way akin to human learning in methodology. When we call it learning that's an analogy. You can not argue a legal case from analogy alone.
I thought this was the most common anti-IP sentiment.
You'll find many more people of the opinion that authors life+70 years is too long, than you will dismissing it entirely.
An extreme example: I do not want my code to be used in weapons guidance systems. Am I not expressing my rights as an author?
If that was not allowed, if people could only use my work for no profit purposes, I agree.
That's what CC (creative commons) is for, though.
I am also in favour of people "remixing" and even making money out of it, but the intent it's paramount to me.
As a long time copyleft activist, I do not understand what limiting authors rights brings to the table, in your opinion.
More than once I contacted the author of some work I liked and they gave me free license to use it, no questions asked (after explaining what I would use it for).
IMO basic cooperation between good willing people goes a long way, without stripping authors of their rights.
Edit: typos
IP is nonsense. It is putting an ownership over an idea, but ideas aren't objects. You can not steal or destroy an idea, giving ownership rights to ideas is absurd.
If you think IP should not exist then creative output is free to use by everyone and of course corporations are also free to profit from it.
over an artifact
You can copy the idea though
And I say copy because before the author made it in the form of an artifact, you didn't know it could be done, or you could have done it on your own.
I can draw ocean waves, nobody owns ocean waves
But if I cooy verbatim hokusai ocean waves, it's mehhhhhhhh
I'm a ripoff loser and should be punished for doing it
Edit: but I like your closing sentence.
Free for all to use it as they see fit is an interesting thought, however I think that imbalance of the means of production will always make the corporations the ultimate beneficiaries, so we should also rethink how our economic system works.
For example: if I could copy the avengers, people would say "it's a cheap copy of Disney's original" and I would hardly make any money out of it.
It's an interesting POV nonetheless.
You are totally free to copy that painting in whatever way you want. There is literally zero intellectual property attached to it. This isn't a hypothetical, it is the law right now. Nobody can stop you painting a replica and selling it, as long as you disclose that it is a replica. It is in the public domain and there will be no punishment.
But I've never said you can't, I've said it's lame and you should be punished for doing it, punished meaning people will totally not attach any meaningful value to it, because it's a stupid replica
Value in art is not given by the content (and usually not the idea), but by the author's intent
The fact that you can legally copy something that is in public domain doesn't take away authors' rights to decide what you can and cannot do with their creation
The problem here is that corporations want to copy creations of authors that are still alive and never granted them rights to replicate them ad libitum, for free.
IP isn't ownership over an idea, it's ownership over a specific artifact.
You cannot copyright the idea of painting or the ideas contained within a specific painting, but you can copyright this specific painting because this specific painting is an artifact.
Do you think a movie's creator can dictate that I'm not allowed to use a pirated version of a movie to display for 100% profit at my 10,000 AMC movie theaters? Or a book's creator can dictate that I'm not allowed to copy their book, put my name on it, and sell it on Amazon for half-price?
If you agree that a creator can do those things, then you're already in a gray area between "they can dictate everything" and "they have no rights." At that point, you're arguing over the precise location of the line.
Yes. It should not. I think that was very clear when I said I was against intellectual property, not for it being more limited.
>Do you think a movie's creator can dictate that I'm not allowed to use a pirated version of a movie to display for 100% profit at my 10,000 AMC movie theaters? Or a book's creator can dictate that I'm not allowed to copy their book, put my name on it, and sell it on Amazon for half-price?
I do not care at all.
Someone's identity, e.g. their likeness and identification aren't just ideas. They actually physically exist. (I assume here that by identity you don't mean how they view themselves). For identities I think it is very important to consider publication. If you publish something you have no right to tell others what to do with it. An actor in a movie has no right to tell someone that their fan cut of a movie is not allowed to exist. On the other hand, as long as you do not publish some part of your identity you should have a right to it. If you are recognizable on a photo that you do not want to have published it should not be published.
To make it short I essentially agree with the GDPR, with the exception of being able to retract consent, but I do not believe it invalidates my stance on IP in any way. Your likeness is not an immaterial idea derived from a creative process, neither is your own house address. You should be allowed to publish it and by publishing it, that publication is no longer yours to own and control.
Author A spends a year writing a fiction book. They are not well known in the industry. They publish the book, it sells very few copies (perhaps none at all).
Person B (a well known Instagram influencer with tens of thousands of followers) downloads a copy of your book. Replaces your name with their name as the author, and promotes it as a book they've finished writing. Goes on to sell at a very profitable rate.
If your answer is NO this should be illegal, what about if Person B searches/replaces all definite articles with indefinite articles, or changes all the proper nouns to begin with "schmla"? Where do we draw the line?
If your answer is YES, then my gut instinct tells me you are neither a writer, composer, or creator in any meaningful sense of the word.
I am an author, a digital artist, a programmer, and an extremely amateur musician. None of those things have demonstrated to me why copyright law is necessary, a net positive for the world, or ethical.
If it's a principles thing—I'd ask, what is good in this principle? What is the good in someone else taking credit for your work? What is the good in someone else divesting you of wages you could use to live or support further creative acts? What is the good in any of that whatsoever? What real harm is the protection afforded you by the IP causing? If you worry underprivileged people can't access your work, you can give it to them for free as the IP holder. I simply cannot fathom any reasonable abstract position in which not having IP would be good for you. Sure, if you hand over your IP to someone else, I could see wanting IP abolished, but that's on you. Thant's the same as the parent scenario only you actually got a say in the matter at least.
I think a very interesting case study would be YouTube. This exact scenario happens there regularly, with various consequences for the creators.
You think this will make the world better? For whom? Or worse, for whom? Or you place the highest importance on having a maximalist viewpoint that simply cannot be argued with, because being unassailably right in an abstract rhetorical framing is most important to you? Or you crave the elegance of such a position, reality and utility notwithstanding? Or you feel the need to rationalize your otherwise unfounded "belief" that piracy and/or training AI on protected IP should be allowed because you like it and are involved with it yourself? Or you think ASI is going to completely transform the world tomorrow, and whether we get Culture-style luxury gay space communism or something far darker, none of this will matter so we should eat, drink, and be merry today? Or some hybrid of that and a belief that we should actively strive toward and enable such a transition, and IP law stands in its way?
What is it? I've seen some version of all of these and frankly they are all childish nonsense (usually espoused by actual children). Are you a new species?
All of human culture is derivative. The current legal regime stifles human expression and makes it impossible for creations to ever be shared in a reasonable human timescale.
To state it in an arguably hyperbolic manner: The "moneyed interests" you're railing against exist because of the current scheme of "intellectual property". They reap virtually all the benefits of human intellectual toil in the system already. Wiping away their stranglehold on the market would be a good thing for creators. Taking away the legal framework their existence is predicated upon would do that.
The copyright industry's influence on social norms, including the massive shift of the social contract in favor of their interests ( functionally infinite copyright terms, attacks of fair use, plundering the public domain to sell it back, works being lost forever because they are "orphaned", etc), all seems natural to you because they want it to be that way. The concept of someone "owning" an idea, which seems perfectly normal to you, was taught to you by people who want the world to be that way, not because it's some natural law. You've been conditioned to believe it your entire life.
I would prefer a fairer system to burning it all down, but the needle has moved so far away from fair that burning it all down seems pretty satisfying.
> All of human culture is derivative. The current legal regime stifles human expression and makes it impossible for creations to ever be shared in a reasonable human timescale.
In the context of copyright law, this is absurd. Fair use exists and it generally isn't even required to "allow" artistic and cultural influence to propagate. If I want to write a sci-fi novel that is heavily influenced and arguably even derivative of my favorite work of some other author, I am completely free to do so---i just can't copy it wholesale. Established fair use doctrine is certainly subject to criticism and future reform, but in a much more nuanced scope than you are applying here.
If you're talking about patents, that's a different conversation. But based on context, that isn't the conversation we're having here.
> burning it all down
Generically, this is another common suggestion from people with naive viewpoints and little or no relevant experience and/or exposure. Notably, people said that about both of Trump's presidential terms to date. I will let you make up your own mind on how that is going for us.
edit: I should also mention that literally every working artist I've talked to about the prospect of abolishing IP law is vehemently opposed to it.
Finding success defending a derivative work on the basis of fair use, particularly if the derivative work is itself financially successful and derivative of a work "owned" by a rights holder with deep pockets, doesn't seem to be very easy in today's legal climate. It certainly doesn't take "wholesale" copying to attract the attention of a rights holder, either. Heck, with the music industry it doesn't take any copying at all to end up losing a lawsuit.
I'll cite the Tolkien or Marvin Gaye estates as good examples among many. I, too, believe it's a ton more nuanced than how you're characterizing it.
> Generically, this is another common suggestion from people with naive viewpoints and little or no relevant experience and/or exposure.
I said "I would prefer a fairer system to burning it all down...". I even qualified my writing as hyperbolic. It seems disingenuous that you'd choose to belittle and make unfavorable comparisons, given both of those statements.
Characterize my viewpoint how you will, but my viewpoint isn't "burning it all down". I will admit that indulging in a little daydreaming about morally repugnant rights hoarders getting what they deserve is fun, but it isn't realistic or productive for society. A fairer system, enabled by shorter copyright terms and better handling of "orphan" works, would be the right thing and what I wish for. Just like I said.
> I should also mention that literally every working artist I've talked to about the prospect of abolishing IP law is vehemently opposed to it.
Well, yeah-- of course they would be opposed to such a vast, sweeping change. It would be madness if they weren't.
The interesting conversation to have with working artists would be about reducing copyright terms, and how that would both "cost" them in potential lost revenue but at the same time free them from the specter of potential litigation and, at the same time, open up vast tracts of public domain work they could build upon.
Believe it or not, there really are people who think literally deleting all of IP law is the right thing to do, and it is my honest belief that my first comment in this topic is in reply to one of those people. I appreciate your clarification of your viewpoint (and the appeal of a fantasy in which we Stick It To The Man by cutting off the hand that grips Mickey Mouse and other beloved IPs by the throat), but I hope you can see why I read your comment as I did, especially in the context of what I was replying to.
> The interesting conversation to have with working artists would be about reducing copyright terms, and how that would both "cost" them in potential lost revenue but at the same time free them from the specter of potential litigation and, at the same time, open up vast tracts of public domain work they could build upon.
I am 100% in favor of reducing copyright terms. I'm not sure I've ever talked to somebody who isn't—I suspect I'd have to talk to a corporation-as-person to find that viewpoint in the wild.
Yes.
(Obviously there is zero point in corporations doing this as costs are near zero and competition for this infinite. Right now almost all art is available for free, it is just illegal.)
>You think this will make the world better? For whom? Or worse, for whom? Or you place the highest importance on having a maximalist viewpoint that simply cannot be argued with, because being unassailably right in an abstract rhetorical framing is most important to you? Or you crave the elegance of such a position, reality and utility notwithstanding? Or you feel the need to rationalize your otherwise unfounded "belief" that piracy and/or training AI on protected IP should be allowed because you like it and are involved with it yourself? Or you think ASI is going to completely transform the world tomorrow, and whether we get Culture-style luxury gay space communism or something far darker, none of this will matter so we should eat, drink, and be merry today? Or some hybrid of that and a belief that we should actively strive toward and enable such a transition, and IP law stands in its way?
I think it will marginally improve the world. I am not training any AIs though.
Could you consider what abolishing IP laws would do to the average YouTuber. Exactly nothing. Their content is available for free, they support themselves without selling their art directly. Loosing rights to their art would have zero impact on them, as their monetization works without it. Sponsoring, direct support, advertisement is enough to make many of them wealthy.
> Obviously there is zero point in corporations doing this as costs are near zero and competition for this infinite. Right now almost all art is available for free, it is just illegal.
This is incredibly naive. You grossly underestimate the grip massive media corporations with essentially unlimited marketing budgets and total control over mainstream distribution channels have over how the vast majority of consumers consume media. And, to whatever extent you think piracy being legal will change that, either way you've completely destroyed individual creators' ability to even partially support themselves with their work, unless...
> Could you consider what abolishing IP laws would do to the average YouTuber. Exactly nothing. Their content is available for free, they support themselves without selling their art directly. Loosing rights to their art would have zero impact on them, as their monetization works without it. Sponsoring, direct support, advertisement is enough to make many of them wealthy.
...Unless you force them all to be social media personalities and marketers first. Unless you think YouTube and its ilk can carry art and culture forward alone (as "content", of course). Unless you want to live in a world where art of original substance is no longer produced, a hall of mirrors in which YouTubers endlessly inter-react and beef and soy face. You may very well think that sounds great, but I think it sounds fucking terrible.
How much do you think I am paying an artist hen listening to their songs on Spotify 1000 times? Every album they ever made, dozens of times. The answer will surprise you!
Already every single musician has to be a personality and marketer. There is no other way to make money. You can not finance yourself by the rights to your music, right now.
Only a very select group of artists has any kind of real revenue from the rights over their music. The rest is already using other channels to profit, which are not protected by IP.
So maybe you'd like to state, for the record, that all those sales, royalties, and other fees—in aggregate—are so small as to be meaningless and insignificant to those who receive them? It's staggeringly wrong, but at least then we'll know where we stand.
Yes they can because the analogy directly applies. The technical details of how a computer learns, vs a human learns doesn't really matter here, and is an irrelevant difference.
The reason why the analogy directly applies is because in both cases it is about how IP cannot really control how someone uses the IP.
Just like how IP laws cannot prevent you from listening to music while standing on your head, it also cannot prevent you from training your models on it (while also standing on your head, lets say!).
Instead, IP laws only prevents the publishing of copies of the IP.
So the point and the analogy stands.
Yes they will. Judges have already ruled that you can't just ban people from using your IP, like by standing on your head.
It is only publishing someone else's IP that is disallowed.
This is copyright law 101.
You can it seriously be arguing that an IP owner could ban someone from listening to their music while standing on one's head, for example.
Instead, they can only prevent copies from being distributed.
Even if we limit our area of consideration to just Copyright, you're just wrong. As the world currently stands, you probably get your music from a streaming service. That streaming service doesn't grant you the right to copy, perform, or adapt the music you listen to, they extend you a limited license. If you are in violation of that license, they likely reserve the right to terminate your access to the music. If that license said you weren't allowed to be upside down then standing on your head would allow them to prevent you listening. It does not matter that Copyright doesn't explicitly mention "listening to music while upside down" because your use is not mediated by copyright, but by the contract you signed.
Copyright does not govern how those individual contracts are formulated. We have Freedom of Contract in most of the western world. It provides a framework by which that contract even makes sense (they have property you even care to license with a contract), and a remedies should you violate it.
So, once again, the point stands.
If the law did not work that way, then the GNU GPL would not be enforceable.
The GPL for example required that if I distribute a derivative of a GPLed work in binary form, then I must also distribute the same derivative as source code or at least provide the source code to any distributee who asks for it. This requirement of the GPL has been enforced by courts (e.g., in a lawsuit against Tivo) even though none of the defendants in these court cases signed a contract regarding the use of the software.
So, in other words, if you don't distribute a derivative work you are fine.
Like I said. The point stands once again. Everything that you said only applies for distribution of copies. And if you don't distribute copies, and you don't sign any contracts, you are fine.
> This requirement of the GPL has been enforced by courts
You have misunderstood that court case. Nobody can be forced to follow the GPL. Instead, you can only be forced to pay damages for distributing copies.
Following the GPL is simply a method of avoiding damages for distributing copies. But you don't have to follow it. You can simply either not distribute copies, or you can pay damages for your distribution of those copies.
So the point stands once again, that you cannot be forced to follow these contracts, and you can only be prevented from distributing copies.
So, once again, the point stands.
It's funny because you can just look at the current legal landscape about how people using these AI models are doing fine.
You can download whatever AI model you want and you aren't going to get in trouble and there are trillions of dollars worth of companies being created.
Yet despite the fear mongering, the government isn't shutting them all down.
The AI companies won. AI is everywhere. And it's not being taken down by the government. Game over.
And this has been obvious for over a year now.
Of course it's different, but if we look closely, it is not copying. The model itself is smaller, sometimes 1000x smaller, than the training set. Being made of billions of examples, the impact of any one of them is very small (de minimis).
If you try to replicate something closely with AI it fails. If regurgitation was a huge problem we'd see lots of lawsuits on output, but we see most suits for input (training). That means authors can't identify cases of infringement in the outputs.
If you are against IP, you obviously are against artists policing which model is trained on their data and which isn't. As this very obviously assigns ownership to the creation of the artist, aka IP.
And generative AI is not a person in the first place, so I don't think the appeal to learning makes much sense here.
Do they? Please cite your studies.
US Constitution, Article I, Section 8, Clause 8:
> [the United States Congress shall have power] To promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries.
I don't believe it's morally right to license music but, if I did, I'd look at the climate of lawsuits in the the last 10-15 years and conclude that the risks might outweigh the potential rewards.
I wish we could have it back. I would feel a lot better about "intellectual property", morally, if we did.
We can come up with new models that compensate creators more directly. We can also come up with models that don't involve effectively killing any ability to build upon existing culture in the time-scale of a human life.
What came before isn't how it has to be, except that the interest that profit from the current system want it to remain so.
Then you have guilds, trade marks, potter's marks, royal warrants, etc... all "IP" protections of their days.
Would open-source license violations be possible to penalize if not for intellectual property laws?
I also never said that artists should not be allowed to do this, just that I hoped their methods were defeated. The same way I hope copyright protections are defeated.
I also pirate every single book I read. Sometimes I buy them though.
I also pirate every single show I watch. I never buy them.
Music is a bit difficult, but I pay for Spotify, but I wouldn't mind paying for the service if Spotify had no rights to the songs and wasn't compensating the artists.
How can you possibly justify this? Do you propose professional artists/authors/musicians just shouldn't exist?
You've been conditioned to believe the current model is the only one that works. I'd argue that this is, in large part, because a bunch of interests who aren't "creators" profit from this current regime.
Yes, artists existed before IP but they also existed before the internet and digital works. Prior to the internet, creative works met real material limits and real scarcity due to the limitations of physical media. In the digital age, these limits are obliterated. You then have two options:
1. You instate something like IP to make digital markets roughly (and admittedly arbitrarily) like real material markets. 2. You establish no such system. No property rights exist in digital space.
I am a major fan of (2) myself, but history gave us (1). At this stage, too many people make a living off digital markets for us to just impose a radical shift—you're taking about reworking the entire global economy here. Without concomitant shifts toward socialism globally this would probably just result in digital space becoming sparse and people adopting "analog only" release models to try and sustain incomes off of creative work.
If you want to abolish IP, go the full mile and abolish the principle of property period. The distinction between IP and physical goods is just an accidental feature of digital technology. If you are against the idea of IP you are against property. Period. And I'm in full agreement, but I also don't think it's easy to achieve this at this historical juncture. Go read Proudhon.
Everything in my post is true. And I do not intend to be inflammatory.
>How can you possibly justify this?
Easy. I buy the books from authors I respect. Books are mostly passion projects anyway and most of the ones I read are from authors who absolutely do not make a living from them or are dead.
The other category of books are scientific publishers I hate all of them and they do not deserve a single cent from me. I would feel bad giving them any money at all. I also am already paying their authors with my taxes.
For shows. I hate most of them anyway and I wouldn't mind if corporate entertainment were gone.
For musicians. I see it the same as with books. I support the artists I like and I give them money for products they put out.
Nobody owes an artist his existence. Especially since most artists are commercial failures any way and somehow that does not threaten the art itself.
Also consider YouTube. Do you think YouTubers would care about IP laws being abolished? Their products are available for free anyways and they commercialize themselves in many different ways.
If you abolish IP youtubers will not be able to make money because if I rip someone's monetized video and host my own ad-free copy, people will prefer to watch that and seeing as there is no legal claim over their IP, the youtuber has no recourse to get me to take down the video. There you go, I just divested a youtuber of a potential revenue stream.
This is why people who favor IP argue that it stimulates the creation of creative works. Look, whether we like it or not, everyone needs to earn wages in our system. If you can earn wages solely on creative works, you are empowered to spend more time and energy on those works because they give you a direct return. If the return on creative works becomes effectively zero modulo charity of consumers, you will have way less resources to devote since now you need to spend them making wages in other ways.
I am all for an IP free world, but I think it requires a fundamentally different economic reality from the one we currently have.
Haha. Do you think that isn't happen right now? Anyways it is irrelevant to the YouTuber, ad revenue is just a small part of it. As blockers are widely in use and YouTubers still exist. YouTube ad revenue is low anyways.
YouTubers make money through merchandise and sponsorships. Merchandise can not be replicated as people buy it to support the YouTuber (selling merchandise while lying that it is to support the creator is fraud, even without IP) and for sponsorships it is irrelevant who also watches the video.
If anything the YouTuber should upload to the ad free site himself to get more reach.
If people can't make money without the government enforcing artificial scarcity of their output, they can always chose another business model.
Is that empathy, or just self interest / good business? If I need a product I can't produce for my business to make a profit I'd better be giving my supplier a cut, lest they stop supplying me. With intangibles it's more fluid than with good bound by scarcity, but novel expressions of creativity become not-novel pretty quickly and, inevitably, you're going to need to go back to the well.
How many paid products are using foss software without paying the developers. 99%? I do not consider that being evil.
If there are no IP laws nobody will have the expectation to have control over what other people do with their creative output. If you are unwilling to accept that you do not get to make art, the same is true now for art as well. If you do not want someone to make a parody of your art your only option is to never publish it.
Imagine I put up a ghetto blaster and a hat. I play the finest music ever made. Would you put money in the hat? Would the idea to put money in the hat ever cross your mind? Would one even understand they want you to put money in the hat for playing the music?
Okay, but this literally goes for every single good ever. Case in point: theft. If theft wasn't "arbitrarily" made illegal, then there exists no business models for anything, ever. Because you could just steal it for free.
The idea behind IP is that IP is a good but an ephemeral one, one without a physical manifestation. So, we need to translate a type of theft that works for that.
Or, we could not. But keep in mind IP doesn't just mean music. It means, like... everything that isn't physical. Including my job, and probably your job too.
For books, most of the ones I read are passion projects and if I like them I buy them as thanks to the author/publisher.
I also have absolutely no problem with paying people for their work, I just do not believe they have any rights to their so called "IP".
Even corporate funded researchers, e.g. for AI, will publish and most of the published results are not protected by patents. Everyone is free to implement someone else's research paper and use it for their own commercial application. Science is also gravitating more to open access, e.g. with arxiv for mathematics and CS. Nobody owns "the right" to some mathematical proof or concept.
Citation is of course encouraged, but plagiarism is not a legal matter, much less a criminal matter.
It's been struggling since the internet became a thing. People got more content than they can consume. For any topic there are 1000 alternative sites, most of them free. Any new work competes against decades of backlog. Under this attention scarcity mode, artists devolve into enshittification because they hunt ad money, while royalties are a joke.
On the other hand, people stopped being passive consumers, we like to interact now. Online games, social networks, open source, wikipedia and scientific publication - they all run in a permissive mode. How could we do anything together if we all insisted on copyright protection?
We like to make most of our content ourselves, we don't need the old top-down model of content creation. We attach "reddit" to our searches because we value comments more than official sources. It's an interactive world where LLMs fit right in, being interactive and contextually adaptive.
Copyright exists to enrich the interests of the publishers of a work, not the artists they funded. A long time ago, copyright was a sufficient legal tool to bring publishers to artists' heels, but no longer. Long copyright terms and the imbalance of power between different wealthy interests allowed publishers to usurp and alienate artists' ownership over their work. And the outsized amount of commercial interest in current generative AI tools comes down to the fact that publishers believe they can use them to strip what little ownership interest authors have left. What Benn is doing is looking for new tools to bring publishers to heel.
IP is fundamentally a social contract, subject to perpetual renegotiation through action and counter-action. If you told any game publisher in the early 2000s, during the height of the Napster Wars, that they'd be proudly allowing randos on the Internet to stream video of their games being played, they'd laugh in your face. But people did it anyway, and everyone in the games biz realized it's not worth fighting people who are adding to your game. Even Nintendo, notorious IP tightwads as they are, tried scraping micropennies off the top of streamers and realized it's a fool's errand.
The statement Benn is making is pretty clear. You can either...
- Negotiate royalties for, and purchase training data from, actual artists, who will then in exchange give you high-quality training data, or,
- Spend increasing amounts of time fighting to filter an increasingly polluted information ecosystem to have a model that only sorta kinda replicates the musical landscape of the late 2010s.
A lot of us are reflexively inclined to hate on anything "copyright-shaped" because of our experiences over the past few decades. Publishers wanted to go back to the days of copyright being a legal tool of arbitrary and capricious punishment. But that doesn't mean that everything that might fall afoul of copyright law is automatically good or that generative AI companies are trying to liberate creativity. They're trying to monopolize it, just like Web 2.0 "disintermediation" was turned into "here's five websites with screenshots of the other four". That's why so much money is being poured into these companies and why a surprisingly nonzero amount of copyright reformists also have deeply negative opinions of AI.
I believe that artists see the current IP laws critically, of course they do as it directly impacts how they finance themselves and how they bargain. But I do not care how good/bad the bargain for the artist is. IP laws should be abolished regardless of what artists want.
Artists rely on some form of IP to help secure payment for their creative works. They need this payment to be able to afford their own subsistence so that they can continue to live and create.
In an alternative system, maybe you could abolish all forms of IP outright, but how will you do that under capitalism while sustaining (already impoverished) artists?
If you are against the principle of IP, you are essentially saying that an entire segment of capital should be deactivated, and effectively the only jobs remaining would be those of active service/tangible goods. In the age of digital media, basically everything is instantly and infinitely replicable, so you are effectively asking for a world in which it becomes rapidly impossible to make money off of any kind of digital good (music, literature, film, software, etc.) This has an obvious material consequence of disincentivizing creation of these works simply because if the creators need to earn wages in tangible good/service markets they have strictly less time to devote to the creation of creative works.
Plainly false. YouTube clearly proves this wrong. You can be funded by your fans.
Besides, if you are a musician and get just tens of thousands of plays on Spotify you are basically not making any money of it. Only extremely few artists make any kind of money of the copyright of their songs. Already distribution rights are near worthless.
Again. I am not opposed to IP rights because of what artists want or do not want.
Nobody owes artists the ability to make a living just for being artists.
Nobody owes artists commercial success for the act of making art.
The idea that it is work theft not to compensate failed artist is ridiculous. Who is to pay, the state?
Of course many of the most successful creators in the world give their creations away for free. They do not charge a single cent from their watchers and still can make enormous amounts of money. YouTube proved that you do not need IP protection to become wealthy from your creations. YouTubers make money in other ways, totally independent from policing their IP.
I didn't say "artists should be given money because they tried hard." I said that if you're profiting off someone's work, you should pay them.
Most of what FOSS license do is explicitly denying the exploitation by someone or some entity wanting to profit from it, without giving anything back.
For example
Unless you have a service industry job or construct material goods, by your own arguments, I see no reason I should pay you either if you work on anything remotely related to intangibles like software.
I'm getting the sense that your perspective is colored more by some personal bias on what art is, what art making entails, and what art is good, rather than any sound logical principles around labor and economics, which is what any reasonable approach to IP should actually be based on.
They don't, what's happening here is their music is being fed to a computer program in a for-profit venture.
This anthropomorphism of LLMs is concerning. What you're actually implying here is that you believe some computer programs should be awarded the same rights as humans. You can't just skip that like it's some kind of foregone conclusion. You have to defend it. And, it's not easy.
I believe that no artist has the right to tell anyone what to do with the art they have published. It does not matter what happens inside the algorithm with that art. Whether NNs (LLMs, by definition are about language) learn like humans or not is totally irrelevant to my point.