Fat pointers are a good idea, but I think it’d be a better idea to store the start and evd pointers instead of a pointer to the first element and a size_t.
If you have a start and end pointer and you move either one, you don't know what the original size of the allocation was.