The experiment used a classical computer to randomly generate a circuit C, told the quantum computer to execute it, and recorded the result. Then they repeated this but executed each circuit in the most optimal way a classical computer can. Finally they compared the distributions to verify that the quantum computer's results matched the correct classically computed results.
This proves that the quantum computer is able to generate that distribution faster than a classical computer and isn't just doing some other spurious process that happens to be faster.
Yes, this random distribution isn't very useful and so this result is only really interesting as an experimental verification of theory. However, it is a computation that benefits from quantum speedup in real life!
Hopefully soon, algorithms will be found to generate more useful distributions (as it seems for the time being sampling is the only application of this type of QC that is practically doable). For example, Aaronson mentions that generating verifiably random bits is not much more difficult than the noise generated in this experiment and could have an impact in a variety of cryptographic applications.