It seems like in order to show me where my missing item is, they have to be able to determine my device’s information. They might claim to choose not to do that correlation until I open the app and go looking, but once I do, it seems obvious that Apple can tell where my device is (in order that they can tell me where it is in their Find My app).
The part you're correct about is that as they control the device, there's nothing saying they can't build a backdoor into it that reports the information (ie. location) back to them once it hits your phone. And we're also taking it on trust that it works the way they say it works, as it's not open-source.
But as someone else commented, eventually you have to trust something.
It's possible in theory they could add hooks into the OS to then do that next time you use the key (e.g. effectively a baked in backdoor) or maybe in some cases the key can be extracted from an iCloud backup or similar (I'm not 100% sure how those keys are stored, but it's likely detailed in their security documentation) but in general the service itself cannot see the encrypted device location.