> all of these concepts become much easier working with "offset from an array" rather than a pointer directly
This reminds me of this article [1] that summarizes very nicely several data structures that represent sparse tensors. I highly recommend it if you need to work with data in two dimensions (like a list of vectors) or more.
In particular, the Compressed Sparse Fiber is a kind of generalization of the "offset from an array" approach, in several dimensions. That can be handy when working with data in 3D for example.