That's not a OS limitation, this is a UX dark pattern from WhatsApp that they've purposefully added to make the UX terrible to push people into granting "Full Access".
I just tested it with both "Add Photos Only" and "Limited Access" modes with Signal and iOS does exactly what you described as the perfect UX. It's literally the following:
1) Tap Add Photo in a chat
2) System photo picker appears
3) Select which photo you want in your entire gallery (not limited to photos previously granted to Signal)
4) Photo is sent to chat
Again, this is with both non-Full Access modes. I think your beef is with Meta, not Apple.