He could have overfit due to his training size being too small. Either he has to add some noise or increase his set.

That's a gene thing, apparently :) https://www.telegraph.co.uk/news/science/science-news/331761...

Maybe your reward function is just wonky.