> while apparently using an open-source app you didn't build yourself does.

Assuming that the app can be reproducibly built, then yes, using an open-source app does give you more confidence that it's secure.

For a closed-source app, you can try to decompile it, which is far harder, or monitor its behaviour, which is far less reliable.