The binary formats that immediately preceded the current OOXML were OLE compound files (aka "structured storage), which is basically a filesystem-in-a-file that's intended to be used for serialization purposes - to allow files to have arbitrary nesting of COM components.
Individual data structures are then binary-serialized into that compound file, but I don't think it's accurate to call it a "memory dump". People often get that impression after looking at the compound file and seeing garbage there, much like unused blocks in memory - but that's because it's a filesystem, and as such, has a concept of unused "sectors". This is also why a freshly saved binary Word file might still contain bits and pieces of old data and metadata.