Because if memory is a big array of bytes and pointers are indices into that big array, that should be numerically valid. And indeed, if instead of making xs and ys separate arrays, one writes:

    int zs[20] = {0};
    int *xs = &zs[0], *ys = &zs[10];

then no perplexing behavior is observed.

Of course, any veteran C programmer will point at that being UB, but the big-array-of-bytes (with holes) model is what one sees in userspace assembly, and what I assume most systems programmers think about most of the time, rather than the C memory model.

I wouldn't recommend that someone trying to learn systems programming (as opposed to C or C++ as languages) spend, like, any time whatsoever memorizing the list of C UBs (and then learning about pointer provenance, exposed addresses, etc., I suppose?). Assembly, Forth, safe Go, and safe Rust don't have the same kind of UB, and (Linux) kernel C has a different set of UBs than userspace/spec-compliant C.