Ask HN: Should the US pass sweeping restrictions on AI development?

The more we are getting ourselves all worked up about how dangerous AI is, the less we are thinking about the fact that LLMs are sucking up all the intellectual property we created in the information age, and then selling it back to us.

Astounding. IMHO the biggest achievement wasn't technical at all--it was how they were all able to do this gigantic end-run around copywrite, trademark, or patent protections.

Essentially, there is no such thing as intellectual property anymore.

AI training doesn't constitute copyright infringement. It's not the same thing.

Could I host a Hollywood movie on my personal website? Could I take the text of "Harry Potter" and sell it as my own novel? If not, then intellectual property obviously still exists.

Inaccurate hyperbole doesn't help your side. It just makes claims about AI training overstepping IP boundaries seem illegitimate.

> AI training doesn't constitute copyright infringement. It's not the same thing.

Why not? E.g., chagGpt 3.5, if you ask what the first paragraph of "Moby Dick" is, it prints it out verbatim. How is that not reselling somebody else's IP?

If you ask it to print out the entire book of "Green Eggs and Ham" it dutifully says it can't because of copywrite restrictions. HOWEVER, you can ask it to print out the nth page, and it will happily print it out, for all 1 <= n<= #pages in book.

It snorked up Dr. Suisses IP, and is reselling it to us.

> Could I take the text of "Harry Potter" and sell it as my own novel?

Well, I asked chatGtp to print out the first paragraph of Harry Potter, and it did, but said that it might violate copywright.... LLMs are quite "self aware" (for want of a better word) that they have appropriated copywrited IP. It will still sell it back to us.

LLMs are a compressed representation of the training set.

So...in answer to your question, no, you cannot take the text of "Harry Potter" and sell it as your own content. But chatGtp sure can!!!!!

> Inaccurate hyperbole doesn't help your side.

sigh yes, its important not to overstate your case. Let me try again: Intellectual Property Laws are for plebs like us. OpenAI and its cohorts, however, have found a way to rip off, repackage, and resell any IP they want to.

And all this fuss over how dangerous AI, and how it should be regulated, etc, is just a false flag to distract us from the real issues.

ChatGPT can't sell it as its own content, ChatGPT has no brain, no thoughts and no agency. There are a few questions that matter here.

* Is the literal act of copying data to the hard drives used in model training fair use?

* Is the act of training using the data count as a performance of the work? (This one likely not because there's no human being performed to.)

* Are the resultant model weights considered a derivative work of some or all copyrighted works in the training set?

* Does a model reproducing some or all of a copyrighted work count as a performance? Does transmitting it as an API response? Does hosting it in the conversion history?

* If the answer to the above is yes who has done copying? OpenAI or the user? What situations is it one, the other, or both?

Nothing says you can't acquire data with reckless abandon, and use it to train your AI which would be fair use. I would love to see someone get sued for piracy and mount a defense that they were torrenting to gather data for AI training and see how it goes.

> ChatGPT can't sell it as its own content, ChatGPT has no brain, no thoughts and no agency. There are a few questions that matter here.

ChatGPT is a machine built and operated by humans. If ChatGPT infringes copyright, then it is those humans who own and operate it who are infringing.

The fact that they built a complex obfuscated system that stores many copyrighted works in a novel way (model weights) and reproduces them incompletely or unpredictably when asked, doesn’t change the fact that they have built a system whose purpose is to infringe copyright.

It’s a little bit like saying I started the car running and let it drive off on its own accord in front of a huge crowd of people, I’m not responsible for the deaths because it was the car that ran them over and the car doesn’t have agency. The car wasn’t designed to kill and never intended to kill. But it’s still me who operates the car and should have foreseen the obvious consequences.

I don’t buy into any of these arguments that just because ChatGPT is very complex or mysterious that somehow the humans who built or are running it aren’t responsible for its actions.

> It’s a little bit like saying I started the car running and let it drive off on its own accord in front of a huge crowd of people <snip>

Interesting. Why do you feel as though this is the proper analogy, rather than being the car manufacturer who builds the car which could be used in this way?

What about a car manufacturer who knowingly does not put in a widget to ensure the car is never allowed to exceed the speed limit? Or does not install a breathalyser which drivers need to blow into in order to start the car?

It's also worth noting that simply reproducing copyrighted material is not necessarily an infringement of that copyright, and in most jurisdictions, it depends on the purpose the reproduction is intended to serve.

I don't think any analogy is needed: if I can go to chatGPT, and have it start printing out pages of a book, that is in itself, ipso facto, indeed tautologically IP theft.

Unfortunately, copyright law is complex, and simply reproducing a book may not always be IP theft. There are many potential ways it could be perfectly above board.

For example, the copyright on the book may have expired. Or it could be fair use (however that's defined in the user's jurisdiction). Or the user of ChatGPT may have a physical copy of the book and lives in a jurisdiction where this allows them to obtain a digital copy.

Sure there's plenty of ways to copy a book and not break the law...but as long as there is any way to break the law by selling copies of the book, then the law is broken, because that's exactly what opneai is doing.

> who has done copying? OpenAI or the user? What situations is it one, the other, or both?

You are way overthinking it. Take my example of "Sam I Am" by Dr. Suess. I can buy access to chatGPT, and With the right prompts I can get it to print out every page of that book.

How is this different from somebody selling photocopies of the book in front of Barnes and Noble?

It doesn't matter what storage method or media is used. You can use a database, an LLM, translate it to swahili and back to english, whatever. If you can get somebody's IP without paying them, then axiomatically, indeed oxymoronically, their IP has been stolen.

>Essentially, there is no such thing as intellectual property anymore.

If only

Imagine no possessions....its easy if you try....

I don't believe you're right, but that would be the greatest outcome imaginable to me.

If AI means the freeing us all from living under the thumb of the current copyright cartel, I'd be dancing for joy. The tiny handful of powerful media conglomerates that sue 12 year olds for sharing a song, take down criticism by abusing automated DMCA systems for obvious fair use and use their monopolistic power to enrich themselves at the cost of everyone else in society should be dismantled.

Of course, in practice, I'm sure the lobbyists from the major copyright profiteers are already working to make sure they get unjustly paid and fair use for regular citizens gets further eroded.

No intellectual property protection means you destroy the next generation of scientists.

It's dumbfounding to watch everyone equate piracy with hollywood movies, while the real innovators and scientists get thrown under the bus by theivery.

It is order of magnitudes more difficult to spend 10years studying something when your work can be stolen and reproduced at any moment.

The world is broken and ctrl c, ctrl v broke it.

Don't expect einstein level scientists and their downstream products that started the computer revolution... you killed the protections needed for the people that made it, cause you wanted movies for free.

It's insanity.

Really not following this. As an ex-academic, plagiarism still carries heavy penalties, and would still be punished if there is no intellectual property, right?

IIRC Youtube was not an authorized "seller" for music videos and movies and Google gave them cheap dark fibre which was no match for many ISPs who complained. Youtube won.

This is a very interesting observation....I never would have thought of the parallels between OpenAI and Youtube, during this startup/rapid growth phase. You are exactly right; YouTube also bent IP laws to the point of breaking while it was trying to grow fast enough to get a commanding market share.

However, eventually, YouTube had to issue endless EULA-violations and copywrite takedowns...and the free ride was over.

Maybe if YouTube had convinced everybody that its tech was an existential threat, governments would still be debating about what regulations should be enacted, instead of enforcing IP laws on YouTube....

Just underscores what a GENIUS move it was for OpenAI to hype-up the existential threat angle. It got everybody scared, and when you are scared, your reasoning is impaired, and you start making dumb decisions. Just brilliant...

You mean learning from humans to help humans?

"intellectual property"

Intellectual property should not exist. All knowledge should be free if it is beneficial to the well-being of humanity. I believe that open source AIs will leak the knowledge from closed AIs, and provide democratized access to all human knowledge for any human who needs it for their well-being and survival. That is something that intellectual property cannot do because it is a mechanism for enforcing scarcity not universal survival and well-being.

If intellectual property has the potential to threaten our survival, and I sincerely believe that it already is doing so, then we must dissolve every single piece of it, and make it a common good for all.

Absolutely not. One has to wonder if the cultural climate was what it is today during the golden age of microchips whether there would be calls to impose a maximum transistor count per area, especially if they could foretell how much power this technology gives to those who master it.

What that genuinely achieves are:

- Power being consolidated to only some small number of companies that can lobby and spend well. This leads market capture and oligopoly. And bars smaller players from entering the market. This leads to poor conditions for consumers similar to aviation industry, cellular networks and broadband market.

- Innovation stops, and growth stagnates. Consumers and developers are left with poorer and higher paid products. The development of the technology stagnates.

- I guess government will also monitor development of weapons, etc. China, Russia won't have similar restrictions, and will get ahead.

> According to this government-commissioned report

I feel like I can guess their stance on the topic without even knowing what it is.

“Outlaw the publication of weights” is this regulatory capture or just the usual gov stupidity?

>Would this still allow AI development to reach a state of usefulness that has been promised for years

It is better that the ban start now because no one has a method for determining which frontier models will become dangerously capable just as no one at OpenAI had any way of determining at the start of training that GPT-4 would be able to get a 92 on the bar exam. That is a pity because the benefits of creating more capable models is considerable: they could've made us all wealthier and saved lives through medical advances.

I think we're probably heading toward another AI winter anyway, so this would be like the government covering it up until things thaw again.

But anyway, there's no way to enforce not running certain programs without some kind of Orwellian surveillance baked in to hardware that I would abhor more than Roko's basilisk, and you now make the people at the forefront of the technology criminals and geopolitical adversaries. It would be an ultra-bonehead move. As always, technology and law don't mix well.

It's just a bunch of cranks who did the report. I hope no important politican takes this "report" seriously.

You're about 1 year behind the White House: https://twitter.com/kristjanmoore/status/1663860424100413440

Yes. The US government should ban the training of new foundational models and should do whatever it can to slow down progress in GPUs because the way things are going now, we are very probably doomed.

Even if China or some other country does not impose similar restrictions, a ban in the US and Great Britain would buy humanity a decade or maybe even 2 decades in which we might find a way out of the terrible situation we are in. (A decade or 2 is about how far the US and Great Britain are ahead in frontier AI research.) Also, buying an extra decade or 2 of life for everyone on Earth is a valuable goal in its own right even if after that Chinese AI researchers kill everyone.

If humanity can survive somehow for a few more centuries, our prospects start improving because in a few centuries, the smartest among us might be smart enough to create powerful AI that doesn't quickly kill us all. It is not impossible that the current generation of humans can do it, but the probability is low enough (3.5 percent would be my current estimate) that I'm going to do anything I can to slow down AI research or preferably stop it altogether -- preferably for at least a few centuries.

How are we doomed? Or what is the doomsday scenario you have in mind?

From what I read we have: massive unemployment, AI making military decisions. What else?

Massive unemployment is not the main problem.

No one has a good idea on how to control an AI that is more capable than people are over a broad range of pursuits. (Actually we don't need to be able to control such an AI, it suffices for it to be aligned with our long-term interests. No one knows how to do that either.)

The Machine Intelligence Research Institute has been paying researchers to think up ideas for over 20 years. The executive director of that institute and its most senior researcher (Nate Soares and Eliezer Yudkowsky) have been publicly saying now for a few years that it is hopeless and that there is no way any of their institute's research directions will bear fruit before AI kills us all. They say that people just aren't smart enough to figure out how to control or align the powerful AIs that will be created soon if things keep on going the way they are going; consequently some sort of pause or halt on AI research -- almost certainly for at least a few decades -- is going to be necessary for us to survive.

The people in charge of the AI labs care enough about their own survival and the survival of their families that they could probably be persuaded to adopt an effective alignment method if someone or some group can come up with one, but it is highly unlikely that anyone will do so in time. Note that whether the effective alignment method is used to align the AI with the broad interests of the entire human population or whether it is used to align the AI to the particular interests of only the leaders of the AI lab is not a particularly important consideration at the current time: it only becomes important in the very unlikely event that someone somewhere finds an effective alignment method.

AI is essentially a subfield of the broader field of optimization; an AI applies an impressive amount of optimization pressure. (The human brain can, too.) When you (or a team of researchers) apply tons of optimization pressure, or create an AI that does so, you steer reality towards some "optimization point" or goal. Most optimization points are incompatible with continued human survival when enough optimization pressure is applied towards that point. And no one knows how to aim a powerful optimization process at the few points that are consistent with human survival and human flourishing. No one has even provided a mathematically precise specification of an optimization point consistent with human survival (and mathematical precision would be quite useful and I suspect also necessary for the specification to be useful to a team creating an AI). The closest anyone has gotten is a 2004 publication by Yudkowsky called "coherent extrapolated volition".

"The hidden complexity of wishes" is one way in which applying tremendous optimization pressure can go wrong, like in the story of the powerful genie who grants 3 wishes to the person (robber?) who found a magic lamp, but the hidden complexity of wishes is not the main snag.

If an AI is better at discovering scientific laws and inventing technologies than people are, it doesn't have to gain control over a nuclear arsenal: it can create its own arsenal that is better than nukes out of materials that people did not even suspect could be used as weapons (just as before about 1930 no one suspected that uranium could be turned into a bomb).

The status quo is and always has been unbearable, and forcing people to exist in it for centuries longer if there’s a better option just seems horrific to me. If there were an agi button that I could press, I’d take a 70% chance of failure, but I personally think that’s pessimistic.

Your use of the term "status quo" suggests a framing in which some corrupt minority has control over our society, and the rest of us must take desperate measures to overthrow this minority if we are to have any hope of leading worthwhile lives. This is a misleading frame (in case anyone is considering using it). You and I are not being oppressed unless perhaps we are being oppressed by reality itself: it turns out that human existence is more precarious and dangerous than we thought and we hoped it was: we can build machines more capable than ourselves. We are well on the way to do so. Every step of the way has proved profitable (and has for example allowed you and I to exchange these words with ease without our having to leave our homes), but the natural conclusion of this path of development is human extinction because humanity is clever enough to create machines more capable than we are, but not clever enough yet to control them.

If we don't pause AI research till humanity is smarter, there is no 30% chance of success. If we look back at history at human projects that at the time were thought to have a 30% or 70% chance of success -- if we look back using our superior modern knowledge of those projects, you see that most were doomed from the start. Consider for example the project of turning base metal into gold. Completely doomed from the start. The alchemists living in the year 1450 did not know that: they thought they had a decent chance of succeeding. Consider most past medical treatments: cutting holes in a mentally-ill person's skull in the hope that that would let the bad humors leave the head; nailing religious symbols to front doors to prevent the house's occupants from catching the contagious disease going around. The thing all these doomed projects had in common is that the people pursuing the project lacked a sufficient understanding of the part of reality they were trying to affect.

Similarly, we don't understand how to align a superhuman AI. Some of us have been trying to understand it for 20 years. Some of us have been getting paid a salary for 20 years to try to understand it. No one has even gotten close. There's no 30% chance: there might be a 3% chance, and if you think life is so bad that choosing a lottery that ends humanity with p = 97% and removes human suffering with p = 3% is a good move, well, most people disagree with you. And (I mean this in all sincerity and kindness) I'm sorry life has been so unbearable in your experience.

Even following all of this very closely it's hard to tell where the ceiling is, there are constant improvements, in 1 year or so we've seen at least 3 generations of significant performance upgrades.

I have not read the report, what are the reasons they recommend such drastic measures?

In summary, AI implemented in weapons systems present a tangible threat, along with AI trained past a certain order of magnitude potentially being adversarial to humans by default while not revealing its alignment.

Lethal autonomy is more than 20 years old in the United States: https://en.wikipedia.org/wiki/MIM-104_Patriot

At this point, I think we can safely say that making a system that gives AI enough control to fuck up is a "skill issue" in earnest. The overwhelmingly obvious solution to this problem is to just not design systems that trust AI whatsoever. Problem solved, alignment crisis averted.

The genie is out of the bottle, you can't just undo the last year of progress and knowledge. Even if the US took this drastic step, there will be other countries that will push forward. Or private interests. Or well-funded terrorist organizations. Etc, etc, you get the point. Much better, in my opinion, to stay ahead of the curve and tackle whatever issues head-on rather than burying your head in the sand, hoping for the best, and pulling out the surprised Pikachu face when someone else inevitably pushes state of the art forward.

Throwing large amounts of compute, power, and learning cycles is building fascinating outcomes. That alone means strategic competitors should be secret.

I genuinely can't understand the doom. Can someone provide me with a realistic step by step scenario where language models destroy human civilization? How can it counter each individual step we take to correct/secure our computer systems? I feel like I'm lost here. There is beautiful mental gymnastics we have to make to conclude that AI will cause the civilization destroying dangers that we all think(want?) to happen. The only thing that I can think of is somehow a language model accidentally launches all the nukes in the world. But, that assumes that literally every step of the process can be actioned upon by typing text into a terminal. There's many armed people and physical keys that are in between GPT4 and the launch codes...

Two likely scenarios:

1. Bad people will use powerful models to do bad things.

2. Many jobs will be automated, and many people will get laid off.

1. Bad people people have been doing bad things literally each and every day since the dawn of man. How does ChatGPT make this different? Do good guys not get the same tools to make the world better and safer?

2. This scenario I can understand causing _upheaval_ and significant societal change, but I can't see it causing the death of the human project like some predict. Will there be more food in this new society where it's incredibly cheap and efficient to make it? I presume so. Is it impossible for us to figure out how to distribute the food? No. Why will humanity come to it's end because people are not required to crunch numbers in excel day after day anymore? I'm not trying to be an ass, I just haven't seen an explanation that makes actual realistic conclusions.

I thought we were talking about why would someone be concerned with recent progress in AI. That’s the question I tried to answer.

2. Regulate. Restrictions on companies using AI to soften the blow - I don't think we're ready as a society to adapt to fast changes and potential mass removal of certain job families.

1. Don't restrict this. Open source will develop AI further etc. Perhaps a knee jerk reaction at how one can use this tech in military applications.

Regulation to stop selfish profits, companies are lusting over themselves to make people's livelihood obsolete.

Capitalism and AI might be a toxic mix in the end.

How can you possibly regulate 2?

No, in an arms race either you win or your laws don't matter.

That’s quite lulzy. But never estimate the stupidity of USG …

God, no. The United States can’t possibly put that genie back the bottle, but it can cement the incumbents’ status as government backed monopolies in the next era of computing. That’s what all this doomer nonsense is really about.