Simple: we don't want some low level kernel memory management dictating what constitutes an "object".
Everything isn't object-oriented. E.g. large arrays, memory-mapped files, including executables and libraries.
Linear memory sucks, but every other organization sucks more.
Segmented has been done; the benefit-to-clunk ratio was negligible.