Hacker News Reader

> Most developers don't realize that standard PyTorch files are just Zip archives containing Python Pickle bytecode.

This is outrageous. Why not deprecate this cursed format and use something from the data frame community? Like, Parquet or something

Actually almost any binary format is better than this

Pickle files are probably still useful saving exploratory work, collaborating inside a company, and use inside a pipeline.

Safetensors is supposed to be the successor for distribution. I believe that it's the "safe" subset of pickle's data format.

The safetensors file format is a header length, JSON header, and serialized tensor weights. [1]