But that's just an implementation detail leaking.
If the GC language properly supported interior pointers (and first class pointers) for the GC there wouldn't be a need to force boxing in all cases. You would be able to make the decision at the point of instantiation of the object. It would make the GC more complex, but I think these days most GCs support interior pointers for other reason anyway, right?
Or am I missing something?