I'm not sure that's largely true. I suspect that many property-owning voters are okay with the concept of public housing, but are less ken on the actuality that building public housing increases the supply of housing which reduces the value of their property.
There's also the weird conservative thinking that public housing provision somehow amounts to people getting "something for nothing".