Apparently, many people on this website think Go and StarCraft qualify as "the real world". Majority of them probably never even heard of blocks world and SHRDLU.

AFAIK, the biggest problem in reinforcement learning still is connecting the final outcome with individual actions you took to achieve it. This hits you hard when you move from toy domains to real-world problems, because you can't replay real-world scenarios thousands of times, and even if you do, you can can get completely different results due to various random and external factors.

This can be somewhat mitigated by taking Marvin Minsky approach and building / using a simulation of the problem instead of the real thing. However, in many domains building a realistic simulation of the domain is significantly harder than supervising learning.