Would've, should've, could've.
Programming languages are a huge waste of time. It's a domain with lots of opportunities for bike-shedding, and then for hard-core R&D too. But programming is still engineering, simple & easy that gets the job done wins over needlessly complex. And yes, inertia is real too, but if a programming language was dramatically better at what C does best, I wouldn't still start new code in it.
When you're doing systems programming, pointers being "glorified" 64-bit ints is pretty much what you need. When your requirements hit reality, all the nice abstractions break down quite easily.
> I avoid raw pointers whenever I can when I write C++
Empirically, this is the path to madness.
> - comparing pointers to different types should be invalid
It produces a warning on my C compiler and an error on C++ compiler. Note that there _can_ be pointers containing equal addresses with different types -- a struct and its first member have the same address.
> - adding pointers to the same type should be invalid
I think it is, or what exactly do you mean?
> - subtracting pointers to the same type should return a signed 'pointer difference' type
it does, that type is called ptrdiff_t, which is a signed integer type.
> - Get the address contained by a pointer with Ptr<T>::address()
But you didn't want them to be glorified 64-bit ints? Well, because of pointer arithmetic, which is a practical necessity, they kind of are, but then again they're not because C tries to abstract from that. You can kinda unwrap the pointer abstraction by casting to the (optional) uintptr_t type.
The thing is, not all machines necessarily have one flat address space that contains everything from functions to all data. Pointer representation isn't necessarily uniform -- e.g., you can have a bigger type for void-pointers that contain type information too, this way you could do run-time type checking.