Julia aims to also be a C-Compatible low-level language with manual memory management in addition to the GC.
When building data-structures you're often using small unsigned integers that use the complete value-space, (e.g. u8s and their bitsets in tries, u16s and blocks with that size in sparse vectors).
With 1 based indexing, you can't do that anymore without first having to cast the index into the next larger integer, adding 1, and then doing the access, which is a PITA, adds instructions, and messes with prefetching, while you're already working with difficult, hard to read, and heavily optimised stuff.