To whom? AFAIK if you self-host, no data is leaked.
EDIT: if the person you're talking to is hosted on Google, Google has all the metadata. Even if one person is hosted on Google and is part of a room you're in, Google has all the metadata in that room.
In 2022 e2ee is the standard for messages but not metadata, and unfortunately Matrix doesn't tackle it, so no, you can't expect 100% privacy.
Would be interesting to know if Amazon could reconstruct that information without Signals knowledge