Mostly, even homeschoolers, aren’t.
They either (and the latter is, I guess, a special case of the former, but worth calling out separately) think the existing public schools available to them are a failure in general or for the specific needs of their children that is unresolvable in time to adequately serve their children without being “against public schools” more generally (I’ve known people who work in public education and homeschool that are in this group with regard to their children’s specific needs), or they are ideologically opposed to education that isn’t guided by indoctrination in their particular religious/ideological beliefs and believe that the current public schools available to them are inadequate on those grounds.