Markets are clearly more efficient at the top (and all but the most inclined wont be running their own hedge funds). Buffetts, Paulsons, etc cannot just move into a position on a whim. They thus require more asymmetric information for a trade to make sense.
Inversely, smaller traders avoid the enormous position sizes that can make transacting difficult, and thus can benefit from increased mobility in the markets. This means less lucrative information advantages can be harnessed more easily. Additionally, there are inefficiencies that simply cannot be realized by large funds (see the Norilsk Nickel tender in 2011, "free" money for small investors: http://kiddynamitesworld.com/norilsk-nickel-russian-odd-lot-...).
I'll temper my entire statement by saying I agree with you for 99% of the population, but I do think people too easily punt on efficient market theory. I do believe the small, inclined trader can profit.