You're using that as an excuse..
The point is that eventually a quantitative difference becomes a qualitative difference. Few people can make a 10% annual return and many, many fewer still can make a 50% annual return. But if you just buy index funds, it's fairly straightforward for just about anyone to make a 4% annual (real) return.
Given the gigantic difference in how these curves grow over time, it's entirely fair to point this out.
It's like this popular meme:
http://i0.kym-cdn.com/photos/images/original/000/572/078/d6d...
Yes the beginning looks easy, yes I like where you got in the end, but you seem to have left out some important details on how you got there...