They also don't have to cater to special education students (although some private non-profit schools do, for sure), and they don't have to subsidize special education funding with normal student funding.

One of the consequences of our state (Washington) going to a state-wide student funding model is that public schools really don't want to lose their non-special education students, because they get a fixed amount of money per student and if they lose too many of the students to private that don't need (and don't get) extra resources, they don't have enough funding to provide programs to the rest.