Zoning is to prevent a paper factory from opening up shop next to an elementary school, not to give themselves a job by creating arbitrary rules.
You've thrown out two contradictory statements. If you agree with zoning in certain circumstances then you can't hold that property is a right for use by owner as they see fit(without the threat of being kidnapped or stolen from). Just because the current zoning system in most locales is overly bureaucratic, or even corrupt, doesn't mean the idea of zoning should be done away, or that property is a right the government can't interfere with.
Criticism is one thing, but framing this as an issue about the government taking away our rights is not productive.
Forbid people from building factories in commercial zones, or restaurants in residential zones without permits, but don't impede the reverse. If people want to live in an industrial zone, let them.
Sure there are downsides to this, a developer could buy up part of an industrial zone and build houses on it, which might reduce the efficiency of the industry around it, and would require the residents to petition to have their area rezone to avoid someone building a paper factory next to their elementary school. But as discussed, there are downsides of the current approach.
This article describes it rather well.