His books tend to be dry and geared towards researchers, in my opinion. He has a new one on RL: https://web.mit.edu/dimitrib/www/RLCOURSECOMPLETE%202ndEDITI...
Very curious about RL for LLMs for example (using data from real use).
Neither cover LLMs. I don't follow the literature closely so I can only suggest you read papers: https://github.com/WindyLab/LLM-RL-Papers