Realistically, it hurts private schools by reducing the number of students they could take on. Public schools, however, are governmentally forced to abide by a certain curriculum, which hurts students dramatically (basic biology and sex education is a big one here).
Because of this, what actually happens is the top half of students that can't go to private schools get an underwhelming education. Compounding on that, local districts can mismanage funds and security to an extent that reduces the likelihood of a student's success in the classroom to near-zero.
Net result - <10% of students get a great education and greater opportunity. 40% come out 'fine' but not particularly ready for entering the workforce or higher education. The rest come out with an education that is measurably substandard, resulting in a significant hardship.
Example numbers, based on your comment.