The language we speak and the concepts we are taught literally form into structures in our brain, and we can't understand what it would be like not to have them, because those structures are part of the organ of understanding
For example, people normally count things by starting from one, so, at least following this usual procedure, counting to zero is technically speaking impossible. Also, we can't distinguish between zero apples and zero oranges, but we can tell two apples and two oranges apart.
In fact, even with the perspective of modern algebra zero remains special. For example, it is the only element of a field without a multiplicative inverse.
I'd be surprised if zero didn't take any extra effort to discover. It's clearly different than other integers.