Because zip codes can, in many cases for many historical reasons, strongly correlate with brown and black folks, which are legally protected classes.

As much as you might think you are being clever by just saying "Oh, I'm just excluding AREAS, not people" you are still running afoul of the law since you are effectively using zip codes as a proxy for race. This has been tried before and the courts aren't really fooled by it.

So, no, there is nothing inherently wrong with excluding certain areas from seeing your ad, unless you are using the area exclusion mechanism as a proxy for excluding based on race/sexuality/religion or some other protected status.