We are already starting to see the nature of data changing. Unsupervised learning is starting to work — see https://blog.openai.com/language-unsupervised/ which learns from 7,000 books and then sets state-of-the-art across almost all relevant NLP datasets. With reinforcement learning, as you point out, you turn simulator compute into data. So even with today's models, it seems that the data bottleneck is much less significant than even two years ago.
The harder bottleneck is transfer. In most cases, we train a model on one domain at a time, and it can't use that knowledge for a new related task. To scale to the real world, we'll need to construct models that have "world knowledge" and are able to apply it to new situations.
Fortunately, we have lots of ideas about how this might work (e.g. using generative models to learn a world model, or applying energy-based models like https://blog.openai.com/learning-concepts-with-energy-functi...). The main limitation right now: the ideas are very computationally expensive. So we'll need engineers and researchers to help us to continue scaling our supercomputing clusters and build working systems to test our ideas.