Hacker News Reader

Comment by jstummbillig | Hacker News Reader

> Distillation “attacks” are not attacks.

If "distillation attacks" happen, we have to conclude there is some value add in what model labs do. Regardless of how we feel about using existing human knowledge in the way they currently do, it's simply impractical to infer that everything that happens downstream of LLMs can not be an attack on some IP because of it.

So both things can be true: a) People infringe on Anthropics IP and b) what Anthropic did to build their models is legally questionable (or might be ruled illegal, even though I doubt it).

View on HN

>People infringe on Anthropics IP

No.

Authors do not infringe on IP when they read another's book, nor should the lumber company be able to dictate how I use planks and if I can resell them if i'm done with them.

You're framing it as if the added value of the author or lumber company, awards them consideration when somebody uses the products to create more value.

IP law was always a big mess, and these questions cross far into ideology instead of law; but I do not understand people who think we need an ideology where more IP-law is good for society.

It's more simple: They infringe on the IP by way of violating the ToS. If you violate ToS and the company suffers financial harm, they usually can (usually) sue you in civil court for damages.

Violating terms of service and violating IP rights are two independent violations. Neither implies the other.

You can't violate ToS you never agreed to. If I use pirate Claude through a third-party reseller, I have entered no agreement with Anthropic.

Api key?

I guess you could steal them but thats a whole other issue.

That’s not what “IP” means. You’re describing breach of contract.

>Authors do not infringe on IP when they read another's book

Are the distillers reading books or are they building models?

If anthropic is providing no value they can just build from scratch. But obviously distilling is easier. Hes saying thats the value they add.

There are some quite interesting legal implications here. If Anthropic has IP over output produced by agents, do they somehow have legal rights to code and documents produced by such agents?

This would demolish agent usage by corporations.

You say “if”. How did Anthropic obtain this IP, if the model serves ripped internet and all human knowledge?

General consensus is that neither the model nor its outputs can be protected IP

> People infringe on Anthropics IP

Unless someone literally stole the weights somehow (which is not out of the question, I doubt either oAI/Anthropic have the capabilities to prevent a state-level actor getting those weights), distillation from generations is not infringement on anyone's IP nor is it stealing nor is it an attack. It can't be. As long as you pay for tokens you get to do whatever you want with them. Someone saying you can't doesn't mean it's an attack or their IP or whatever. They either sell the tokens or not. They can decide to not sell them to anyone, but again that's not stealing.

And their ToS are a joke. Imagine how people would react if MS had ToS saying that you can't use MS software to develop solutions that compete with MS. They'd be laughed out of the room. Somehow it's ok for token sellers to decide what you do with the tokens? Why? If you pay for something you get to do whatever you want with that output. Train, distill, whatever.

Its definitely an attack. Thats established from anthropics perspective. No one has a right to use Anthropic’s services in ways that directly violate the ToS and user agreements.

> Its definitely an attack. Thats established from anthropics perspective.

How do things get "established" from someone's perspective, exactly?

By that logic it is established from my perspective that Anthropic has no right to train on anything I've written that is publicly available on the internet.

Of course, they don't care about my perspective, but then again I don't care about theirs.

There's a lot of people diligently shilling for Anthropic in this thread. That's established from my perspective.

> from anthropics perspective

I guess I can see that, if you mean the targeted effort of creating many accounts w/ the intent of doing it at scale. Sure, they may see that as an attack. But again, it's only an attack from their perspective if you agree that using generations to distill is "wrong". I just don't see it, in general. You can't both sell tokens and decide that distilling is somehow illegal. Something, something, cake and eat it.

The output of Anthropic's models is not Anthropic's IP, as that would destroy their market, if Anthropic owned all the software it generated, and all the content. So distillation, which is just using those outputs is always going to exist.

> People infringe on Anthropics IP

Anthropic’s model outputs contain no IP. This is actually a simple legal proposition (rare in this field!) that derives from the fact that only specific classes of IP exist: copyrights, patents, trade secrets, and trademarks. Examining each, it is clear that API outputs do not qualify. Anthropic disclaims copyright in outputs; the outputs are not patented; the outputs are not secret (a prerequisite to having trade secrets); and trademarks are irrelevant in concept.

I'm pretty sure that LLM output is not intellectual property. Nobody owns it, and it can't even be copyrighted. So using output from Anthropic's LLMs in ways Anthropic does not condone is not IP infringement.

Whether or not its legal to distil models, it is obviously morally permissible to do so.

Anthropic, OpenAI, etc do not deserve legal protection.

anthropic model output is not their IP

that would be existential doom for them because then they have a case to claim ownership of their users' codebases

no corporation would sign off on that

The value is simply that it is easier. The same way it is easier to ask someone who has experience for advice than reading hundreds of textbooks.

Considering they were the original infringers, I don't know how anyone can expect tears to be shed here. The best we can hope for is for all these cancerous - and they really are the definition of a cancer - money burning entities to all fall apart to distillation attacks like these.

Anthropic’s IP is basically null and void for how they created it. And they might not want to try and challenge this in court, considering how they had to settle for using text books they had no right to use

What IP is being infringed on? AI outputs aren't copyrightable.

Regardless of whether it’s intellectual property or it isn’t intellectual property, it doesn’t actually matter. If AI doesn’t stop seeing diminishing returns in scaling up, and it hasn’t yet in the 10 years since the attention/transformers paper, the advent of AI will be the most important development in the history of humanity. Controlling that machine, or at least having one of your own, is an existential problem for nation states. It’s like a matter of national defense.

Do you really think intellectual property laws will prevent this in practice? It’s like as if we said, “hey, USSR, you can’t make a nuke, too! We patented that already.”

Asking China to not distill our models down is equally as ridiculous.

I do like that you mention diminishing returns, because we are hitting them in building out all the external requirements for competing at the frontier. Even if model performance scales linearly with energy input, the top labs are now competing with other uses for that energy.

How far are we willing to go as a nation (and as a species) to prove out the scaling laws? Are we willing to sacrifice our industrial base? Would we rather train models or smelt aluminum?

It's unlikely the USA would be granted an exclusive patent for the atomic bomb given the well-established existence of prior-art in the form of nuclear fission on the sun.

(I actually appreciated your analogy, despite my lark)

In fact, China stealing fire from the gods is essential to the future balance of power, so long as they keep making the results freely available.

Reply on news.ycombinator.com