Hacker News Reader

Comment by saosebastiao | Hacker News Reader

I know ML is the big cheese right now, but doesn't it seem like a bad use case for the cloud? Consider:

1) Training ML models does not require network access, which is one of the biggest competitive advantages of the cloud.

2) Training ML models is typically a batch process, which benefits minimally from the scale-on-demand model of the cloud.

Since the cloud premium is a significant exchange for the value that it adds, I don't see this being a big win for cloud providers. I can't help but think that if I were making use of extensive machine learning with continuous training, I'd have it training models on a local bare metal cluster statically scaled to my application's demand with minimal network connectivity needs. And then ship the serialized trained models to the cloud. The potential cost difference is huge.

View on HN

> does not require network access

> benefits minimally from scale-on-demand

Neither of these is true when dealing with terabytes of data (or more, if you're working with image/video corpora). Many AI/ML problems have stages that are trivially parallelizable - if you can divide your problem into iterations where a subgraph of nodes communicates internally, then sends/receives updates to other subgraphs, it's very similar to an iterative map-reduce algorithm, perfect for networked cloud systems.

And as you're tuning your hyperparameters, you don't know what the performance characteristics are, and you will absolutely want to run experiments in parallel, until you find the right settings that you'll use in production. You'd need to invest in a LOT of redundant bare metal to have that capability. As Netflix puts it in this presentation, the key to effective machine learning is iterating often, and that means having a lot of parallelism to bring to bear. https://www.infoq.com/presentations/machine-learning-netflix...

Good point. I've worked with lots of local exploratory machine learning (R or Spark on a single workstation), and lots of stable applications that use machine learning processes that are no longer fiddled with, but I've never witnessed the transition between the two.

I don't follow point 2, batch processes can definitely take advantage of scaling on demand. You can spin up a huge cluster, only pay for it for an hour while training, and then spin down to just what you need to serve requests for your already trained model.

Of course it depends on how much data you're training on, and how up-to-date you need your model to be.

Makes sense - though if you add some kind of validation to the trained model you could build a continually improving 'ground truth' and automate the training and improvement over time.

Reply on news.ycombinator.com