Hacker News Reader

Comment by ninjin | Hacker News Reader

Certainly looks malicious, but what on earth would be his motive? He is an MSc student for heaven's sake and this tarnishes his entire career. Heck, he has published multiple first-author, top-tier papers (two at NeurIPS and one at ICLR) and is on par with a mid-stage PhD student that would be considered to be doing outstandingly well. The guy would (is?) likely to be on track for a great job and career. Not saying he did not do what was claimed, but I am unsure about any motive that fits other than "watching the world burn".

Also, what kind of outfit is ByteDance if an intern can modify (and attack) runs that are on the scale of 256 GPUs or more? We are talking at least ~USD 8,000,000 in terms of the hardware cost to support that kind of job and you give access to any schmuck? Do you not have source control or some sort of logging in place?

View on HN

> but what on earth would be his motive

Rumors said that his motivation would be to just actively sabotage colleague's work because managers decided to give priority on GPU resources to those who were working on DiT models, and he works on autoregressive image generation. I don't know what exactly was his idea, maybe he thought that by destroying internal competitors' work he can get his GPU quotas back?

> Also, what kind of outfit is ByteDance if an intern can modify (and attack) runs that are on the scale of 256 GPUs or more?

Very high. These research labs are basically run on interns (not by interns, but a lot of ideas come from interns, a lot of experiments executed by interns), and I actually mean it.

> Do you not have source control or some sort of logging in place?

Again, rumors said that he gained access to prod jobs by inserting RCE exploits (on unsafe pickle, yay, in 2024!) to foundation model checkpoints.

Thanks, that is at least plausible (but utterly stupid if true) and tells me why I would not be a good cop. Holding off further judgement on the individuals involved until we have more details.

I do understand that interns (who are MSc and PhD students) are incredibly valuable as they drive progress in my own world too: academia. But my point was not so much about access to the resources, as the fact that apparently they were able to manipulate data, code, and jobs from a different group. Looking forward to future details. Maybe we have a mastermind cracker on our hand? But, my bet is rather on awful security and infrastructure practices on the part of ByteDance for a cluster that allegedly is in the range of ~USD 250,000,000.

Agree on this being stupid.

> my bet is rather on awful security and infrastructure practices

For sure. As far as I know ByteDance does not have an established culture of always building secure systems.

You don't need to be a mastermind cracker. I've used/built several systems for research computing and the defaults are always... less than ideal. Without a beefier budget and a lot of luck (cause you need the right people) it's hard to have a secure system while maintaining a friendly, open atmosphere. Which, as you know, is critical to a research lab.

Also,

> from a different group

Sounds like it was more like a different sub-team of the same group.

From what I heard I'd also argue that this could be told as a weak supply chain attack story. Like, if someone you know from your school re-trained a CLIP with private data, would you really think twice and say "safetensors or I'm not going to use it"?

A lot of ML outfits are equipped with ML experts and people who care about chasing results fast. Security in too many senses of the word is usually an afterthought.

Also sort of as you also hinted, you can't exactly lump these top-conference scoring PhD student-equivalents with typical "interns". Many are extremely capable. ByteDance wants to leverage their capabilities, and likely wants to leverage them fast.

Basic user separation is not asking much though, or are we expected to believe that at ByteDance everyone has a wheel bit at a cluster worth many many millions? Let us see what turns up. Maybe they had a directory with Python pickles that were writeable by everyone? But even that is silly on a very basic level. As I said in another comment, I could be wrong and we have a mastermind cracker of an intern. But I somewhat doubt it.

That level of security is true for most big tech companies :) You mistake thinking that large and well funded = secure. They clearly have an audit trail but no preventative controls, which is sadly the standard for move fast environments in big tech.

I think we are converging at an opinion. Internal actors can be hard to detect, and honestly there is a reason at places like Google interns are treated with heightened security checks (my guess -- they learned to do so after some years).

Btw one of the rumors has that it is even difficult to hire engineers to do training/optimization infra at one of those ML shops -- all they want to hire are pure researcher types. We can imagine how hard it will be to ask for resources to tighten up security (without one of these incidents).

Reply on news.ycombinator.com