Hacker News Reader

I believe you have confused training and inference.

Training usually uses large amounts of data to get your system to recognize a pattern. It generally uses huge memory and compute and generates a model.

Inference will use that generated model to recognize the pattern in new data. It uses significantly fewer resources and can run either very fast or on a smaller system.

I believe the speed of inference may be affected by the resources available during training, where more speed/memory for training can produce better models.

I think it's parent commenter who is confused, complains how hard it is to "squeeze state of the art models into my limited memory", which sounds like inference.

Yeah when you've a trained model you can copy it even to mobile devices.

Very, very much depends on the model.

This would depend on the model

Most ML models can be shrunk at the cost of some accuracy, but some models are extremely large.