Hacker News Reader

Comment by biomcgary | Hacker News Reader

I am not a Pythonista, but joblib is such an easy wrapper to use.

I use joblib at work (it handles edge cases that pickle does not) to package up trained ML models for deployment by a colleague. I train the models outside of a Python environment, pull the models into the Python version of the ML library I use, and then serialize the models to a compressed joblib file. The joblib object takes up less space than the text based model file and deploys quickly for almost instant predictions.

To ensure that the input fields for training line up with the input fields for classification/regression, we add a hash to the joblib object, which is trivially easy. Dump the object on s3 using the hash as the key and it's trivial to have a library of ML models ready to deploy very quickly.

View on HN

At my previous job, we used joblib + a handbuilt file format to persist ML model blobs across versions.

The file format was a zipfile with:

- a joblib dump of the python class wrapping the model and its data processing pipeline code

- a small csv (~100 rows) of test X data, test Y data and the expected outputs of the model on that data

- a little json with important library versions (pandas, numpy, etc.) so that the process loading the code could check it ran the same as the one that produced the model blob

Then the loader would try to load the file and verify it, and if failed would keep the existing model in memory and raise an alert to slack, so engineers could fix it and reupload a new model without downtime.

The whole thing was inspired by the talk "Alex Gaynor: Pickles are for Delis, not Software - PyCon 2014" [1]

At my current job we don't need such infrastructure because lambda/kubeflow gateways take care of it for us.

[1] https://www.youtube.com/watch?v=7KnfGDajDQw

I'm curious what edge cases you know about or have bitten you with pickle. The main ones I know about are that it's not blazingly fast (though cPickle mitigates this a bit), and, as a possibly related issue, pickle actually uses multiple incompatible serialization protocols that are subject to change whenever a new Python version comes out.

It's also insecure in the sense that you can pickle and unpickle more or less arbitrary Python objects and object structures, so there's no way to trust a pickle from a source you don't control, unless you want to start down the rabbit hole of encrypting and signing your pickles. But that's actually never come up for me in practice.

The incompatibility issue has always been the bigger deal for me. You don't want to upgrade Python and then have to deal with converting all your model weights to a new serialization format, too. An unstable data interchange format is absolutely an oxymoron, and all but rules out the idea of using pickle in production for much of anything. The best use I can think of is for RPC between clients you control and trust, but I just can't bring myself to have that level of trust, even for a service or host that I'm sure is fully under my control.

TBH, my knowledge of pickle edge cases is limited to my colleague's recommendation. ChatGPT seems to be able to construct a plausible list that matches several of your points, but who knows if the rest was hallucinated.

Great to hear. Scikit-learn also typically uses joblib for model persistence: https://scikit-learn.org/0.18/modules/model_persistence.html

Joblib is also used internally for models to be run in parallel, e.g. https://github.com/scikit-learn/scikit-learn/blob/1834cd6b76...

(End users typically just pass n_jobs as a non-zero/one value to use it).

Note that joblib serialization is pickle based and therefore has the same security implications as for any pickle file: consider loading a joblib or pickle file as running a compiled executable: never do it if you do not trust the source.

A new safer alternative for scikit-learn model persistence is skops:

- https://skops.readthedocs.io/en/stable/persistence.html

It makes it possible to trust a list of types of Python objects that are safe to load and refuse to load skops files with untrusted types.

Also note that nowadays, with Python 3.8+ and pickle protocol 5, it's now as efficient to do:

  import pickle

  with open("model.pkl", mode="wb") as f:
      pickle.dump(trained_model, f, protocol=pickle.HIGHEST_PROTOCOL)

  with open("model.pkl", mode="rb") as f:
      trained_model = pickle.load(f)

pickle from the standard library with protocol 5 can store and load large data buffers often found as attributes of scikit-learn models (typically large numpy arrays) without extra memory copies (as joblib.dump and joblib.load were designed to do with a few hacks that violate the official pickle protocol).

For reference pickle protocol 5 was specified and implemented as part of:

- https://peps.python.org/pep-0574/

and also provides extra API to handle large data buffers externally ("out-of-band") via custom callbacks. This is in addition to the no-copy semantics memory optimization when loading/storing such arrays "in-band" without providing custom callbacks.

Reply on news.ycombinator.com