They are ZIP archives... of XML. That's what the 'x' stands for in '.docx', '.xlsx', '.pptx', etc; it is Office Open XML[1].

[1]: https://en.wikipedia.org/wiki/Office_Open_XML#:~:text=Office....