Hacker News Reader

Comment by Yoric | Hacker News Reader

If it's PyTorch, it can definitely contain and execute arbitrary code.

One of the reasons I'm not a huge fan of PyTorch.

They could contain arbitrary code... But typically do not. That means that with the right viewer application it will be trivial to know for sure.

It isn't like a multi gigabyte game for example, where knowing if there is any malicious code could easily be a multi-month reverse engineering project to get to the answer of 'probably not, but we don't have time to check every byte with a fine tooth comb'

I only found this picklescan[0] serving this purpose, but it doesn't seem to be a finished project.

[0] - https://github.com/mmaitre314/picklescan

Curious how long that would take on 200gb.

In theory, this could be done, sure.

In practice, who's going to bother checking the language model? All the code that runs Stable Diffusion or other Hugging Face models that I've seen just downloads the model dynamically, then uses it without asking question. That's a pretty low-hanging supply chain attack waiting to happen, I believe.

It shouldn't be much work to verify that the file is just a set of floats.

I mentioned this above, but Trail of Bits has a tool just for this purpose called Fickling. https://github.com/trailofbits/fickling

Anything that loads pickles from sources your unsure of can contain executable code. There were a few samples a couple month ago showing distribution on huggingface.

Some solutions for checking: https://huggingface.co/docs/hub/security-pickle

or run them in an isolated env.

Reply on news.ycombinator.com