You don't need reinforcement learning when you have basic physics | Hacker News Reader

You don't need reinforcement learning when you have basic physics

npdeep.github.io

16 pointsnpdeep9 comments

This is of course a kind of a toy counterexample. But still, it does seem to illustrate the fundamental problem with a lot of ML-based solutions:

"Just because you can, doesn't mean you should"

I mean yeah - it is kind of neat (and interesting, in a fundamental way) that we can just "throw" a learning algorithm at an ever more, and ever broadening class of problems. But from a basic engineering perspective - it doesn't mean you want to. For a whole bunch of obvious reasons.

Then again, if your company leadership has gurgled the kool-aid, and decided that "our customers want an AI solution, because it's like, new and shiny" ...

That's what I feel too. I am a big fan of machine learning but I just wish it wasn't the first thing people jump to.

I’m more and more convinced we are living in a prestige economy. 10 lines of code and you’ve added to your prestige. No college education necessary, you don’t have to compete on kaggle, you don’t have to make anything useful in comparison to the competition. Just prestige in starting a car, saying you can make a car and walking away.

Reminds me of a post on here in 2018, "It Takes Two Neurons to Ride a Bicycle (2004)" [1], where many of the commenters rightfully pointed out that instead of turning this into a machine learning problem, you could just implement a PID controller [2] to solve the problem. With control theory you can model a dynamic process like this more directly, and the theories have been around for a long time already.

ML shouldn't be applied where it's not necessary.

[1] https://news.ycombinator.com/item?id=16215130

[2] https://en.wikipedia.org/wiki/PID_controller

If you simply secure the pole to the cart using a $2 bracket and screws from the hardware store, you don’t need any of this fancy programming. Checkmate, tech nerds.

The problem is the fact that beyond rudimentary linkages, first principles is pretty difficult to model without additional complexities or time being spent on perfecting the state equation. That's where RL or whatever else is new on the block come in to save time & needing reduced expertise. By this rational you could also throw PID at many problems & claim its faster for a basic task.

I’m excited to see basic physics win Dota2.

I guess the whole idea of reinforcement learning is to "learn" how basic physics works from the observations

We can hardcode a solution in a simple example but how do you write a rule to drive a car from pure physics equations?

Indeed, if you already know everything you don't need to learn anything.

Reply on news.ycombinator.com