I went to gifted and talented programs, and even there I felt like the pace was slow and easy for the top kids. If you put those kids into classes with even less academically apt, less focused children, those top kids are going to be bored out of their mind and wasting their time.
Do we truly want to waste the time of our most bright young minds? Do we want to squander their potential when they pick up 1+1=2 in 10 seconds, but it takes other kids 10 minutes? Do we have to force our teachers to make 4-5 different lesson plans, 1 for the lowest, 1 for the highest, and a couple others for the mid-tiers? We're already asking too much of teachers, give them smaller classes with less of a range of aptitude.