At first, Android needed faster phones that it was getting from OEMs, but they went way overboard. Samsung soaks up a lot of that excess horsepower with bloatware.
Here is a $200 phone with 4(!!!) cores and a very robust GPU. Even that is probably a bit more than neccessary. Android 4.4 got some performance tuning. Once ART is the default VM for typical OEM builds, it should be even less apparent why you need a monster CPU chip.
The only thing a thumping CPU gives you is the ability to ignore what your bad apps and OEM bloatware are doing to performance longer than otherwise. Disable the boatware and clear out the sketchy apps you don't use and that Tegra 3 is just fine.