Hacker News Reader

Comment by virgildotcodes | Hacker News Reader

Here's a probably stupid question - if someone were unbounded by ethics and conceivably had enough power and connections to power to shield themselves from many consequences of their actions - and that person owned these DCs, could they in theory observe all the streams of tokens coming in and out of these models, and even exfiltrate copies of these models wholesale to have their own teams do what they will with them in the pursuit of building their own competitive models?

Or is there something fundamental in the way these models get deployed (encryption or something or than legal contracts?) at this scale that prohibits the owners of the infra from gaining this level of insight / access?

View on HN

1) The situation you described would be covered under the contract between Anthropic and xAI, and that any violation of that would be subject to financial penalties and legal proceedings. The US has a robust corporate legal system, and disputes do get resolved through the court system, although in a slow and costly manner.

The contract can stipulate a penalty at a high enough amount to discourage this behavior.

2) Output from models & intra-datacenter communications can be encrypted if customers truly cared.

3) There is no reason do this, because there are far better ways to exfiltrate data from Anthropic models. Chinese companies are already doing this at an industrial scale where they are reselling Claude tokens for 10-20% of the cost while retaining the data to train their own models. https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens...

If we look at Deepseek V4-pro, created by Deepseek who Anthropic formally accused of harvesting Claude tokens at scale, it performs the same as Claude did 6 months prior.

I guess there's a reason why those Chinese companies are in china

Under US law, LLM outputs are not under copyright. Even if those companies were under the American jurisdiction[1], the worst thing they would be guilty of is a breach of ToS.

1. OpenRouter is based in New York, and offers a 1% discount when users enable logging of their inputs/outputs; it doesn't take a genius to figure out why they might incentivize that

> The US has a robust corporate legal system

Thanks for the chuckle. ;)

It does, though.

It does until you're embedded enough with the surveillance system [1]. If a company was able to get immunity in the wat AT&T got it, no contract would protect the other side.

[1]: https://en.wikipedia.org/wiki/Hepting_v._AT%26T

That case has nothing to do with contract law.

And the govt protects its citizens and God is real

Yeah. Well.

I guess as a European I want the legal system to be robust in terms of protecting the citizen’s interests against corporations. I can see how you can also want robustness in terms of protecting corporate interests against citizens, so, yes, the statement remains technically true in terms of robustness.

In terms of justice, again as a European, I think of Justitia, the Roman Godess, who judges based on “ideals of truth, fairness, honesty and justice“, “applied without regard to wealth, power, or other status”, evaluating “the goodness of the heart”. I don’t see these values represented much in the US so-called “justice” system.

Both aspects sufficiently documented for example over and over again in Matt Levine’s financial reporting.

None of those have anything to do with robustness.

Huh?

Cambridge Dictionary: “the quality of being strong, and healthy or unlikely to break or fail”

I provided two dimensions in which I consider the US justice system to have failed. I can provide more. You of course can prioritize different dimensions, but that doesn’t invalidate that I am also talking about robustness. Robustness is not an absolute, but always an adjective - related to a specific aspect.

The top-level discussed a theoretical risk of data being exfiltrated by third parties for training. Other comments discussed contract law and legal remedies, including Delaware’s excellent court system for handling business disputes.

None of this means you don’t have the same rights as an individual to enforce contract terms. I think the top commenters discussed corporate law because that is where most IP theft occurs, and corporate customers have strong contracts covering data protection.

When I chuckle because of the idea that one sees the US corporate legal system as “robust”, and then give an explanation for why I (me! personally!) chuckled, what is the point of trying to argue? For what exactly?

To give yet another personal reason for my chuckle besides the ones I already gave: I don’t consider a legal system where I can simply move state when I don’t like the law not very robust. Or one that can “look back to (a mere) 200 years of case law” - especially given how that is being (re)interpreted in recent years.

That is my personal opinion. You’re entitled to your opinion, your definitions, your perspectives. I remain chuckling :) we don’t have to always agree or dig holes of rational debate, we can just find stuff funny or not. I volunteered why it is funny to me when that causes irritation with some. I read the high number of upvotes as potential agreement.

It's an accurate statement. The US (specifically Delaware) has a world-class corporate law system. Delaware has a court dedicated to only corporate and commercial disputes, with 200+ years of case law.

Man, good thing no one’s ever done something like reincorporate Tesla and SpaceX in Texas after loosing a court case in said Delaware court and having a pissy fit about it.

The state of incorporation doesn't often matter in interstate contracts. Usually they will include a choice of law and forum selection clause specifically to avoid this kind of thing.

I hope the Chinese keep harvesting Claude etc. since they stole all their data anyway, who cares?

> There is no reason do this, because there are far better ways to exfiltrate data from Anthropic models. Chinese companies are already doing this at an industrial scale where they are reselling Claude tokens for 10-20% of the cost while retaining the data to train their own models.

I think you missed the part where Anthropic stopped displaying their thinking tokens over the past few months, and instead now provides “summarized thinking”, letting Haiku summarize Opus’ thoughts.

So it is now much more difficult (impossible?) to distill the models.

I also think you over-estimate how well the legal systems works in the US nowadays, and under-estimate just how much power Elon has in the government.

At this scale thinking tokens don't matter anymore.

In Feb Anthropic called out three Chinese labs for "distillation attacks", but a lab missing in their post actually had most Claude generated tokens among all Chinese labs in their midtrain data :p

Incidentally, Claude started the "Summarized Thinking" bullshit around a year ago.

Deepseek kept its pace of improvement nonetheless.

It's already in the public domain (thanks to the OpenAI trial) that Grok distilled OpenAIs models. Listening to the data going into the models in the data centre would be very similar thing. There's some downsides (you're passively listening, not controlling the queries), and some upsides (way more data). But it only ever gets you to some percentage of the existing production model. It doesn't get you what Musk wants - an AI company capable of designing and deploying leading edge models. It gets you to fast follower status.

xAI had a lot of negotiating power here because Anthropic had ~0 comparable options and ultimately desperately needed the compute now. So, it wouldn't surprise me if data sharing was an explicit part of the agreement

There's accusations that the Chinese labs have done essentially that to OpenAI and Anthropic and exfiltrated their models without having DC access, so if you had DC access, yes, you could do that. If you had DC access though you could just copy the model onto an SSD.

The Chinese labs have been accused of training on elicited chat logs on a massive scale in violation of ToS. That's possibly a real concern, but it's nowhere close to "exfiltrating" the model or even roughly matching its behavior.

Bear in mind these data centers were built for X.ai to use themselves, so there would have been no reason to backdoor them. Also, this is all off-the shelf equipment: Dell & Supermicro servers with NVidia compute modules and ethernet switches.

The fact that X.ai build it for themselves means that they are in full control. It's not challenging to install a rootkit on hardware under your control. Also if traffic within the datacenter is unencrypted, you can probably get copy of all comms via switches.

It's generally far easier to install spying software under a computer under your control than to detect it.

You could, but Grok is pretty high up there, it might not be "#1" but its definitely up there with the giants, people seem to overlook it. Gemini has a similar problem, it was #1 once, and it seems like Google isn't hell bent on chasing #1 they just want to keep iterating over time, they know they just need it to be "good enough" and they'll keep having repeat customers.

If Elon REALLY wanted to do anything like that he would be better off poaching talent from competitors, less legal hell to go through.

If I was training a top-tier model, having a competitors’ weights feels like it would be an excellent refining tool. As a user I find cross-model iteration to be a huge power-tool, and presumably the boffins can zero in on areas of relative strength and weakness and work out which area they want massive amounts of synthetic data from.

No Grok is not 'up there'. Not by any stretch of the imagination is it anywhere close to anthropic or openai in any single domain whatsoever. Not even close to deepseek. It really isnt a good model. Their research team's talent is just not good, unfortunately.

> in any single domain

I hear that it is very much up there in _one_ domain. You know the one.

You must have missed the part where he did poach all the top talent from Google and OpenAI, and then they all quit because they couldn't stand working for him.

Could they also, if they owned a 10,300 vehicle fleet of orbiting satellites, add radar or optical surveillance to each, and track every human on planet early in real time?

Why would they need to do that, when the majority of those humans are already carrying a radio transceiver with them at all times.

What prevents a data center operator from reading your chats? [FEATURE] Provide a way to select your data center #56916 https://github.com/anthropics/claude-code/issues/56916

Given Musk's previous antics, this strikes me as a relevant line of reasoning.

How do you trust space infra? You can't show up and audit some kind of airgap. Whatever encryption keys you have are likely accessible in memory sometime.

Yes he could do that and I thought about this too. Very weird tbh

Reply on news.ycombinator.com