Hacker News Reader

>> as soon as really convincing yet nasty deepfakes of senators and representatives start making the rounds and spreading on Facebook, I suspect a new law will get passed incredibly quickly.

No. This isn't a new area of law. Legal "Deepfakes" have been around a long while. There is even a supreme court case covering the matter: Falwell, 1988. That involved a fake interview with Billy Graham, using his image and putting words in his mouth. If the 1988 supreme court wouldn't protect the reputation of a national religious leader, they aren't going to do so for politicians today.

https://en.wikipedia.org/wiki/Hustler_Magazine_v._Falwell

This is from the movie, but it is accurate to what was said. This interaction really did change supreme court procedure, introducing a far less formal back-and-forth oral argument. Jokes were almost unheard of before this case.

https://www.youtube.com/watch?v=MeTuNES82O0

The trial court argument: https://www.youtube.com/watch?v=TsvB61mDoG8

From [1]:

> In an 8–0 decision, the Court ruled in favor of Hustler magazine, holding that a parody ad published in the magazine depicting televangelist and political commentator Jerry Falwell Sr. as an incestuous drunk, was protected speech since Falwell was a public figure and the parody could not have been reasonably considered believable.

The post you're replying to explicitly talks about convincing deepfakes. Also, Falwell was a televangelist and political commentator, not a senator. So I don't think this court case is a good example.

[1]: https://en.wikipedia.org/wiki/Hustler_Magazine_v._Falwell

> If the 1988 supreme court wouldn't protect the reputation of a national religious leader, they aren't going to do so for politicians today.

Well that gave me a good chuckle! The fact that a supreme judge is appointed by the sitting president and confirmed by the Senate means that they can push whatever dogma the ruling party has.

No, deepfakes are relatively new. It's a specific term meaning a fake video produced with ANNs, not any type of picture that's been faked or photoshopped. The case you reference is a just and paste job, not a deepfake.

I don’t think the method of producing the fake is important in terms of court rulings.

I didn't say it was, I simply corrected the definition of a word.

It kinda is though, if the network is trained on the source material itself. For example if I take the source material and move a few pixels, is that still the source material? What if I flip the video, which is technically moving every single pixel, that's still the a copyright issue even if the video is mirrored. The network is basically taking the pixels of the original content, learning from it, and creating a filtered version.

I think you misunderstood my point. I meant that it doesn’t matter if someone modified it by hand or if it was done using a neural network.

> I kind of feel like that's going to have to be the outcome at some point in the near future -- in fact, as soon as really convincing yet nasty deepfakes of senators and representatives start making the rounds and spreading on Facebook, I suspect a new law will get passed incredibly quickly.

Ugh, that is the type of free-speech infringement that people should be worried about, not all this stuff about how YouTube taking down videos is government censorship.

A really good deepfake is still fiction. Really good writing is fiction. Really poorly-written fake "news" blogs "reporting" lies as truth is fiction. These are all allowed to exist (in the USA) because of the 1st Amendment. It is the price that we pay for living in a free society, that the government cannot tell you "your fiction is not allowed."

Textual accounts of real events, and lies about real events, have existed for ages. Sometimes the lies are more convincing than the truth. Audio recording and doctored audio has existed for ages. Photography and doctored photos have existed for ages. Video recording and doctored video has existed for ages. We'll survive another flavor of doctoring just fine without the government's help.

No, copying and 'reporting on' / parodying / sampling etc. are very different things.

A writer who makes a book that is word-for-word the same as another book is infringement.

Making a pixel-perfect copy of a photo, from scratch, is an infringement.

Copying a song, perfectly, is infringement.

Whether something is a 'lie or the truth' - or 'which one is more convincing' doesn't really matter.

With 'deepfakes' we're going to have to get serious because people will literally be putting words in other people's mouths, creating fake representations of other people, it's going to cause problems. This issue may not actually fall under copyright.

https://en.m.wikipedia.org/wiki/Richard_Prince

I feel like you've replied to the wrong comment - I'm not defending copyright infringement, misuse of celebrity likeness, or the content of this video.

I'm specifically addressing the notion that the government would carve out a free-speech exception for "deepfakes", like "deep fakes of Politicians / Celebrities are illegal." Which would be clearly unconstitutional, because you are forbidding creation of a creative, expressive work.

Whether the works that you are free to create, regardless of format (photoshopped image, deepfaked video), violate some other law - libel/slander - is separate as well. Yes, they certainly could, just like a photoshopped image today could be libelous.

Pop stars on YouTube ? Fair use & freedom of speech ? Deepfakes will be used to cause a war in the near future where people are killed and who are we going to point our DMCA take down requests at then ? What will the third world victims of some hate story stirred up by AI doctored footage care about first amendment protections in America ? Deepfakes are a clear and present existential danger to ourselves and others and at this point I'm afraid you are correct, there is no good solution. God help us all once we can't even believe what we see anymore.

They can already cause a war just by lying about certain classes of weapons being "found" so I don't think deepfakes will help much in that regard.

You can't believe what you see right now. I remember watching people doing insane things on Facebook back in 2011 but it was all fake. Nothing has changed.

>but it was all fake.

You mean like scenes shot with actors to imply certain events happening? ex. scenes protests in some city when it's actually just a bunch of actors making it look like a protest is happening?

Exactly. Propaganda and misdirection has been a constant for a few thousand years.

The harder lesson is learning to catch yourself and see past it before getting manipulated.

The hardest lesson is not knee jerking and giving those who want to control you the tools to limit everyone's speech to just approved material. (China, Iran, Vietnam, North Korea, these ring a bell? VERY slippery slope.)

> Deepfakes will be used to cause a war in the near future where people are killed and who are we going to point our DMCA take down requests at then ?

Wars are started based on disinformation, black propaganda and/or false flags anyway.[0] Nobody (at least who wasn't already spoiling for a fight) is going to start a war just because of a deepfake posted to Youtube.

> What will the third world victims of some hate story stirred up by AI doctored footage care about first amendment protections in America ?

This video was taken down, rightly or wrongly, due to American law. It's quite likely deepfakes are actually 'more legal' in some countries outside the US, due to differences in legislation.

For example, depending on political/sexual content, an Iranian or North Korean video host may be less interested in taking down a deepfake of Jay-Z than Youtube.

> Deepfakes are a clear and present existential danger to ourselves and others

The real danger with deepfakes (and other technology that the rich and powerful don't like, such as encryption) is overreaction and people tripping over themselves to call for even more limits on their own free expression.

> God help us all once we can't even believe what we see anymore.

And God help us all when we have no way to speak out any more. As shown in the link below, we never could believe what we see.

[0] https://www.globalresearch.ca/the-ever-growing-list-of-admit...

I think you're ignoring an approximately 5,000 year history of humans using propaganda, misinformation, and outright lies to manipulate each other. Nothing new here except perhaps a little broader perception of the level of misinformation actually generated.

We (ok, mostly those in power or who want to be) constantly feed each other the most manipulative, self interested lies for personal/organizational benefit, 24/7/365.

Life has always revolved around either being manipulated, or knowing that that's the point of what you hear, and putting in the effort to find out more before forming an opinion.

Hence investigative journalists, etc etc.

> God help us all...

Umm, we've all been deceived about almost everything since forever. There's no sudden worsening here, just a newer level of verification required.

A little broader? You mean reaching everyone on the planet for almost no cost? This is a huge difference, it used to be only nation states and huge actors could spread and maintain disinformation on this level, now an individual can do it with a GPU.

One set of shared delusions is not the same as an anarchy of delusions.

It didn't take many people to create the Spanish American War. The news papers were not states or even large actors, though of course they did have a platform that most others did not.

Gotta fight fire with fire. Use a neural net to detect deep fakes.

http://www.arxiv-sanity.com/1811.08180v3

Major benefit: eventually, all the news networks will be able to decide that instead of whatever each candidate said, they all actually agreed that Medicare for all was the right choice, and that it will be the center of their platform.

And once elected, they'll be stuck doing it, or the news networks will decide that it did happen, and no one will be able to figure out that it actually was not. At that point, is there any difference?

So the law will be whatever Rupert Murdoch (or Mark Zuckerberg) says it should be? That's not a world you want to live in.

> Really poorly-written fake "news" blogs "reporting" lies as truth is fiction. These are all allowed to exist (in the USA) because of the 1st Amendment.

Not if it’s libel. And a convincing deepfake that harms someone’s reputation could easily be interpreted as an act of defamation by the creator, which would mean it’s not protected speech.

Yes, certainly. And that's somewhat my point - the freedom to create these fictitious works is already limited by existing laws, we don't need new ones specific to this medium.

There's no reason to think libel laws wouldn't apply to deep fakes the same way the apply to a photoshopped image, or a textual lie.

> I genuinely wonder if deepfake technology will actually result in new copyright restriction law. I.e. to make it a crime to produce unlicensed deepfakes that are genuinely indistinguishable to the average person, regardless of whether they're parody or not. (While "bad" parodies like SNL will continue to be protected as always.)

You don't need new law. It's already against the law to copy someone's appearance in an attempt to impersonate them other than fair use.

I think you’ve misunderstood the point of impersonation laws, they are about stealing someone’s identity, this is not that... anyone watching these videos can see that it is NOT Jay Z in the video.

There's also defamation laws. I'm sure a lawyer could make a good argument for banning a deep fake, or suing for the existence of one if it causes defamation.

This isn't about impersonation but your copyright over your own likeness that can't be used without consent. Take a game like FIFA or Madden, they have to negotiate with the players (or their unions) in order to create realistic digital counterparts (including the names) and arguably better games often find themselves unable to compete because of this.

> anyone watching these videos can see that it is NOT Jay Z in the video.

I'm not that familiar with JayZ and it's more than enough that I would have been convinced and not given it a second thought. It's not obviously parody and the only tip off it was even fake was "(speech synthesis)" in the title.

Is it really copyright? Based on what I know of it, it doesn't seem like copyright would apply. Trademark I could sorta see, if they registered their likeness.

Quite a few places recognise "personality rights" [1] although the details vary from place to place. Sports games tend to comply so they can be sold everywhere.

Some form of personality rights make sense to me, as otherwise Mickey Mouse would enjoy more legal protection than you and I. And obviously it's not an absolute right - you can't sue red light cameras for photographing you.

There's also the question of whether a neural net trained on a copyrighted image or video is a derivative work - in contrast to a 3D model made by an artist who merely references copyrighted images or videos.

[1] https://en.wikipedia.org/wiki/Personality_rights

I was replying to a comment saying there are laws against impersonating people, so just changing the context because you think it’s not obvious that these are deep fakes creates a straw man to argue with. I’m not sure I believe you that you couldn’t understand that these were AI generated speech either.

> no court would ever uphold this removal, because it obviously falls under "parody" which is protected

Not a lawyer, but...

It doesn't seem like parody to me. It also doesn't seem like it has much of anything to do with copyright at all - one of the videos taken down was reportedly a performance of an excerpt from Hamlet, which is definitely in the public domain.

On the other hand it does seem like a possible violation of trademark law, but I have no idea how that would play out in court. Which is confusing, because I thought the DMCA only applied to copyright (it's the Digital Millennium _Copyright_ Act, after all) but perhaps I'm wrong and it includes provisions about trademark as well?

> Which is confusing, because I thought the DMCA only applied to copyright (it's the Digital Millennium _Copyright_ Act, after all) but perhaps I'm wrong and it includes provisions about trademark as well?

As far as I know, a legally enforceable "DMCA takedown" can only be about copyright infringement. However, YouTube is under no obligation to not honor takedowns that aren't legally binding.

However, YouTube is under no obligation to not honor takedowns that aren't legally binding.

That's what's ripe for abuse. Why isn't it a fiduciary responsibility to their "partners" to make sure they aren't needlessly deprived of revenue, in situations where a 1st year law student could tell them there isn't a legal basis for the takedown?

Details matter - "takedown that is not legally binding" does not mean that "there isn't a legal basis for the takedown".

The legal basis for the takedown is the language in the Youtube terms and conditions that says that Youtube can take down the content at their discretion. It does not matter if that decision is caused by a non-binding takedown request of a self-asserted copyright owner, or by a drunken manager randomly clicking to remove videos, the legal arrangement between Youtube and their "partners" does not assume any fiduciary responsibility to ensure that your videos will be kept unless absoltely needed, and no current law is forcing them to assume that responsibility.

I mean, the statement "why isn't it a fiduciary responsibility to their "partners" to make sure they aren't needlessly deprived of revenue," seems very weird from the perspective of law - why would there be such a responsibility? Do you have some law in mind that would cause it? E.g. CEOs have fiduciary responsibility to shareholders, stockbrokers have fiduciary responsibility to their clients due to specific laws, but in general there is no fiduciary responsibility between parties in a contract, the parties can be 'hostile' towards each other as much as they want.

The default legal position is that every private actor is free to act as they want for their own benefit as long as they are not breaking any laws or contracts. If I cause you some losses by doing things I am entitled to do, there's no possible liability - in order to claim damages, I have to have done something wrong. So in the absence of a specific law or contract requiring Amazon to ensure that their "partners" aren't needlessly deprived of revenue, that lost revenue is not Amazon's problem, and they're entitled to (for example) intentionally cause partners to lose that revenue if it benefits Amazon in some way and there's no statute or contract that restricts Amazon from doing so.

Because the law treats the "partner" relationship as something that was freely negotiated between equal parties, despite the obvious power imbalance. Video creators need to from a union to protect their interests, but since they're all "temporarily embarrassed millionaires" I doubt that'll happen.

Let's not forget that YouTube's ContentID system is a direct result from them being sued by Viacom et all: https://en.wikipedia.org/wiki/Viacom_International_Inc._v._Y....

If YouTube stops taking down these videos based on this "fiduciary duty" and they get sued again, will this union pay for the legal fees and potential damages? Yeah, not likely.

Anyone can sue anyone for anything. YouTube were protected by the safe harbour rules and won a summary judgement on those grounds.

If the lawsuit was so obviously invalid (despite the first summary judgement being successfully appealed), what's your theory on why YouTube settled and agreed to spend money building Content-ID?

I also am not a lawyer but from what I understand the situation is actually more severe than that.

I couldn't find the reference offhand but I've read that YouTube has no ability to adjudicate the validity of a DMCA takedown request at all. They must responed as proscribed by the DMCA.

Which makes sense because once the DMCA is involved it's ultimately a legal dispute between the person that uploaded the video and the person that claims to own the copyright. And that kind of dispute is something only the courts can decide.

Isn’t there some recourse? I think the content maker can assert it’s non-infringing, and then YouTube can host it again without liability: the liability falls on the content maker if it turns out to actually be infringement.

DMCA is not involved in the standard YouTube removal request process - the removal request does not have to fit the criteria of DMCA, the response does not have to fit the criteria of DMCA, and as long as both Jay-Z and Youtube are satisfied, the DMCA does not get involved at all, no matter what the poster of the removed content might wish.

If Jay-Z was not satisfied with the process, he could submit a DMCA request and YouTube would have to follow the DMCA requirements in processing it, but the current process is better for both of them (not for the posters of the contested content, though).

It's not obvious to me that it's parody. Is it actually making fun of Jay-Z? Is it parodying his songs? His voice? It's not a caricature, it's not commenting on Jay-Z's character or vocal styles or lifestyle or opinions.

Whether or not a deepfake is used for a parody is far less important than whether content using deepfakes identifies its artificial nature. Deepfake content masquerading as real content is actually dangerous and should be regulated as such.

But no, Jay-Z should not be able to take down parodies on YouTube that make it clear to the viewer that they are using deepfakes.

Deepfake used for parody is almost certainly protected by the first amendment under US law. So legally that is a very important distinction. (Morally, well, that's not a productive thing to debate on the internet usually)

If it’s convincing enough to make most people believe that it’s real it’s not a parody.

Plenty of people are convinced that Onion articles are real, but they're still clearly parody.

The definition of parody is somewhat vague, and parody is often subtle. It seems difficult to define and enforce objectively.

The youtube definition is "Whoever has the most money is correct"

I imagine that would likely fall under a new trademark or fraud law rather than copyright.

I don't get it. the videos are still here and I can access them just fine:

https://www.youtube.com/watch?v=iyemXtkB-xk

https://www.youtube.com/watch?v=m7u-y9oqUSw

was this an automatic takedown and maybe they have meanwhile been made available again?

> no court would ever uphold this removal

You mean enforce? Youtube can remove whatever they want; they don't need to be in the right about it.

These concerns should better be handled by other kind of law such as defamation, not the copyright or patent or trademark. The exact definition of defamation varies between jurisdictions so what constitute to be the defamation is another problem though.

I believe the DMCA defacto REQUIRES youtube (and others) to work like this. Once the DMCA take-down comes in, if they leave something up, they become liable if the court case goes against them. So if 1000 takedowns come in, and you correctly leave 999 videos up as fair use etc, you still get demolished on the other 1. Meanwhile the company requesting 999 incorrect takedowns owes you nothing.

This is a key problem with DMCA, it shifts all the risk/cost of deciding cases onto platforms, their only logical response is to take things down, so everything gets taken down.

The DMCA process also includes a counter-claim provision: the uploader can simply contest the takedown and have the content reinstated.

But YouTube doesn't follow the DMCA process at all, they have their own procedure, with no such recourse.

My understanding is that if a uploader does that, youtube has to decide whether to keep it up. If they do, they can then be sued by the right holder if it does infringe. If they take it down, no one can sue them.

So that's a huge incentive to just take everything down.

Effectively the whole point of DCMA (this part at least) is to pierce the common carrier protection platforms have.

Am I wrong? If so, any idea what Youtube don't just let people counterclaim and leave everyone else to argue it out in court?

Under the DMCA (technically Section 512), they have to reinstate access to the content at least 10 days and at most 14 days after receiving the counter-notice, in order to keep the safe-harbor (they have to wait 10 days to give time for the rights holder to decide if they want to sue the uploader).

> any idea what Youtube don't just let people counterclaim and leave everyone else to argue it out in court?

Essentially YouTube in the pre-Google days was purposefully uploading infringing videos, and Viacom (and others) used that to sue them and under the settlement agreement forced YouTube to build the Content-ID system and have a more lax takedown system.

>After all, the end result is no different from a really good vocal impersonator. In fact, a really good impersonator will definitely do better.

Really? I thought if it was "so good" that people don't realize it's a parody, it loses the relevant protections against defamation and infringement on likeness rights/image/trademark/whatever.

If that were the case, then people could claim that any parody was "too good" to get it removed.

Parody must include a component of criticism or commentary. If it's just imitation, it's not fair use.

It's a very fuzzy line they'll need to draw. The Onion seems like a good example of parody done really well, to a point that people often "eat the onion" and believe it. Would it be defamation if you published a quote that wasn't actually said by someone? That might be the analog they use to go after deepfakes in the future.

The article doesn't go into detail on what happened.

If it was a [manual] Content ID claim, there's no legal reason YT has to remove the video; the content ID system exists to provide an (arguably abusable) way for YT to scan every piece of uploaded content for copyrighted material.

If it was a DMCA, YouTube has to remove the video to maintain impunity.

But, as stated in a related comment, this is new territory in the eyes of the law. Maybe this video is a parody, but if it's not clear about that it would be clearly illegal. Even an extremely good impersonator can be sued if they try to actually pass themselves off as someone else.

I wonder if there's any legal grounds in the USA for trademarking the sound of your voice, and thereby protecting it from "counterfeiting".

> I.e. to make it a crime to produce unlicensed deepfakes that are genuinely indistinguishable to the average person

automatic Turing test pass

I could actually see it as libel. You’re leading people to believe the artist did something (created a work or said something) which isn’t true. SNL and other parodies would obviously be exempt because there’s no misrepresentation. A reasonable person doesn’t actually believe Trump went on SNL and happens to look like Alec Baldwin that day.