DirectByteBuffer would allow direct pointer access to the backing memory. When used with allocateDirect, you can rely on Java's GC to manage the memory. Not sure if it's a perfect fitnfor the use case but it's simpler to reason about.
This is why some libraries hacks into DirectByteBuffer to deallocate memory explicitly, bypassing the finalizer altogether. For instance, the Netty library has implemented such a workaround: https://github.com/netty/netty/blob/795db4a866401aa172757b95...