Hmm, I can confirm that Signal does work the way you describe.
It looks like there is a separate API for "Private Access to Photos" that is less common than the UX I describe (WhatsApp, Reddit, Twitter, Discord).
Maybe one thing we can agree on is that if apps have to opt-in to the API that's better for users, then we can also blame Apple.