Everything in "learning" follows a good parameterization of p(y, x), the joint distribution of the data, whether unobserved or not. If you have that, you can get everything else.
The core idea of representational learning is important, even though it's obvious in hindsight. When doing statistics, many people assume the parameters governing the conditional distribution p(y | x) are distinct from those governing p(x). So if you're just interested in predicting y, you don't need to model p(x) to get the posterior distribution p(y | x). Representational learning suggests that the parameters of these distributions are coupled. It's saying that if you understand the structure of the world -- you have a good representation of p(x) -- you can make better predictions p(y | x) with less data. This makes a lot of sense because the distinction between x and y is totally arbitrary.