When there is zero of something, what is there zero of? That question becomes a lot more difficult to answer. In theory you could say there is zero of anything not observed, but this isn't a very precise or useful definition and isn't what's done in practice. The things we choose to actively describe there being zero of depends on knowledge of what existed previously, on cultural and linguistics norms, on context etc.
For example, I can say "there are zero apples in the basket", but this requires me to know that there previously existed apples in the basket as opposed to oranges, or that the addressee of my statement was expecting apples to be there. This knowledge wouldn't be required if there was at least one apple.
Using zero fundamentally requires more mental reasoning than using small positive integers.