It's confusing to Americans. To us, "public school" means "government school". We are also confused by "public house".
When learning English in France, you are taught about these British naming quite early and it is always a surprise for the children.
It is not even that there are specific needles for theses schools, rather the fact that a public school is not for the public.
There are also complex unwritten rules governing dress, speech and general manors that serve the same purpose.
There are some job interviews you better not turn up to wearing the wrong colour shoes or shirt.
Personally, I blame the Normans.
This is not true. Public schools pre-date modern state schools and what made them "public", by the original definition, was that they were open to any child - provided that their parents could afford the fees.
Other private schools at the time were open only to the children of parents who met certain criteria, and hence were not "public".
In other cases, "Public [entity]" = "paid by the people".