This "algorithmically deciding what to focus on (exploration/exploitation)" idea is also discussed in the following book:

Algorithms to Live By: The Computer Science of Human Decisions (2016)

https://www.goodreads.com/book/show/25666050-algorithms-to-l...