Hacker News Reader

Comment by robbmorganf | Hacker News Reader

Other comments point out valid technical concerns, but even if those were solved, there's still not a clear application in manufacturing. If it has already been automated (e.g. semiconductors, welding car frames) then there's not a lot to gain from Deep RL.

On the other hand, if it hasn't been automated yet, who is going to invest hundreds of millions into a Deep RL system (never mind an army of robots) when they can just pay a slightly higher variable fee for humans. For example, sure a robot could theoretically sew jeans for $0.20 vs a human at $15.00 (living wage in US), but does any one firm have the scale to justify $500M to train Deep RL?

View on HN

Nike has a market capitalization of over $200B. I’d guess if they could do away with human labor for $500M they would just do it. Cut labor costs and eliminate their sweatshop labor practice PR problems in one fell swoosh.

Remember that Nike only manufactures a small amount itself. Most is contracted out to factories overseas with different incentives.

Nike does their manufacturing in sweatshops. Probably already cheaper than robots and technicians would cost.

Sewing is hard problem for robots but they are already here and promise prices similar to low wage countries: https://softwearautomation.com/

> I’d guess if they could do away with human labor for $500M they would just do it.

I see what you did there :)

It seems to me that the $500M you quote is an exaggeration. I might not understand what it takes to train Deer RL at this time, but it consistently seems that with time, technologies like this are becoming cheaper and more-user-friendly. If not now (you may be right), isn't likely that in the near future we could have tools easy enough to chain together that people after an intense bootcamp could code up a useful Deep RL architecture to produce a useful-for-profit model?

GPT-3 reportedly cost $10M, and Deep RL is notorious for requiring way more epochs than supervised learning tasks like transformers or CNNs.

And that's not even accounting for the cost of robots (which will need frequent repairs when the untrained policies crash into physical objects), salaries for your engineers and all the raw materials that you're going to have to scrap because the robots didn't make saleable product.

I would assume that automating a previously manual industrial process is a lot more difficult than just training a neural network. My gut feeling is that 500M is an underestimate.

You probably have more experience to estimate price better, but I guess I'm working off a different premise. I thought the claim to defend is that "if you go far enough into the future, it will be cheap".

A parallel - facial recognition today is a matter of `npm install face-api.js` (or some other library). Advances in hardware and ML architecture design can bring the currently-challenging world of RL to similar levels.

Another way to put my argument is through a question: do you think that even 20 years from now, the cost of a useful RL system will be around $500M (inflation adjusted)?

There are a couple ways to think about this. Yes, automation has occurred, often at the level of a single machine. There is a difference between automation and optimization. And there are multiple levels to which both can be applied. One area where most previous optimizers fail, and where deep RL can make real contributions, is in the coordination of larger systems. An analogy would be the teams of agents that OpenAI trained to win at Dota 2. Similar teamwork can produce better results in industrial settings than they are currently achieving by automating or optimizing the behavior at a machine-level. The trick is to optimize groups of machines, entire plants, and networks of plants (e.g. through planning where to send items to be processed).

In the world of supply chain and industrial control, people typically fight hard to get an additional 1% gain in optimization. Deep RL can often surface decision paths that lead to double-digit gains, and those gains, for many companies, will be worth a lot of money.

Fwiw, it does not cost anywhere near $500M to train deep RL.

Big companies with big markets, they’re the ones with the money who can take those risks and collect the rewards in the future. That’s why the “big get bigger” and that’s been true since forever.

Reply on news.ycombinator.com