A (Long) Peek into Reinforcement Learning | Hacker News Reader

A (Long) Peek into Reinforcement Learning

lilianweng.github.io

160 pointsBrysonbw16 comments

Lillian's writing is really crisp without compromising the technical details.

I learnt reinforcement learning from Udacity course. It was one of the finest on the internet. Whenever I couldn't find some thing in the course I went to this website. I created my own small introductory course for RL, a pure hands-on experience for learners.

I found this in the Authors GitHub repo. Looks interesting. Will use this to complement my learning resources https://github.com/farhanhubble/discover-drl/blob/master/Red...

If you're interested, this is the course that I created: https://github.com/KnowchowHQ/rl-in-action/tree/master/C1-RL...

The other one you found is just one of the notebooks from the course.

Thanks a lot! Looks like a lot of fun. Like I said will combine it with my Stanford online course

Is anyone aware of a course that is beyond the basics and is focused on modern algorithms?

It sort of depends on which direction you want to go in. If you're interested in deep RL as applied specifically to LLMs, I'd second another commenter's recommendation of Spinning Up from OpenAI. It hasn't been updated for a while and is a little outdated, but it provides a really nice introduction to some key ideas like PPO: https://spinningup.openai.com/en/latest/user/introduction.ht...

If you want to get more of a bird's eye view of modern deep RL in general, then HuggingFace's course is a good place to start. The course itself is kind of "all over the place"—not necessarily a bad thing, just maybe not what you want if you're looking to go super deep on a single topic. But if you want to get a look at some robotics stuff, deep Q-learning, multi-agent stuff, etc., it'll give you a nice sort of "tasting menu." As with most HuggingFace stuff, the format is really nice, it does a good job of introducing you to key ideas/projects in the ecosystem, and it has some cool project components: https://huggingface.co/learn/deep-rl-course/en/unit0/introdu...

I used SpinningUp as one of my main sources for my thesis. I also wrote some downloadable-and-runnable blog posts here: https://avandekleut.github.io/ppo/

This one looks pretty good: https://arxiv.org/pdf/2412.05265v2, although I've only skimmed it so far.

Very nice! I also recommend the CS885 course from university of Waterloo which is all on YouTube. It covers Bayesian reinforcement learning which is missing from most courses (including this one).

Here a link to the lecture playlist: https://www.youtube.com/playlist?list=PLdAoL1zKcqTXFJniO3Tqq...

If you want a (long) peek at RL read Sutton and barto. Amazing book.

(2018)

Has RL changed since then?

This page covers the basics which are pretty timeless. You'd want to understand this before learning about any more recent advances.

It doesn't get into state of the art algorithms, for example proximal policy optimisation isn't mentioned although the paper on this was published in 2017 and is probably considered the best algorithm for at least some applications.

Curious what people here have built using RL?

A mate of mine built a works scheduler using RL + MCTS. It was interesting seeing the scheduler get smarter as they added in reward for real life constraints. For example, certain types of work couldn't happen on a tuesday - they add that in to the reward calculation, retrain, it now avoids Tuesdays. Build up that reward calculation based on available data, and it got to be super capable at making a workable schedule. Also orders of magnitude faster than linear solvers (albeit without guarantee of "optimality").

Highly profitable trading systems.

Reply on news.ycombinator.com