0 in normal usage is always a negation of something existing. You don't start counting things that exist at 0 (as you pointed out in your example of number of red balls in a bag that have none). However, position zero at an array in zero-indexed languages is the first position within the array.
There's no getting around the fact that higher level languages inherited 0 by counting byte offsets, rather than positions, even if those languages don't required that every index within an array have the same size in memory (or occupy a contiguous block of memory), and that this inherited usage is at odds with the actual plain-language and true meaning.