This post is restricted to the context of the European Union and is intended to be factual.
The EU age verification app is intended to be a pilot to the EU Digital Identity Wallet (EUDIW), which EU law requires to be deployed everywhere in Europe by the end of 2026. (Thus your "worry" is in fact the explicit plan of record.)
The EUDIW will store more attributes than age. Think of it as a digital form of a passport (with name, address, etc.). The exact set of attributes is determined by local laws.
Thus, the DOCUMENT that you obtain is tied to you, and of course the state knows what is in the DOCUMENT since the state creates the document in the first place.
The state does not generate proofs. The phone generates proofs. Given a proof (and only the proof), nobody can associate the proof to the phone or to you.
Now I switch to less factual statements, which are still approximately correct.
Why would you trust the wallet software not to phone home to the state or us (Google)? The EUDIW regulations require that the wallet software be open source. However, states will only issue DOCUMENT to their own certified wallet software---you cannot just take the open source and recompile it, since the state won't issue DOCUMENT to your uncertified wallet. (Maybe your gym will issue a gym membership to your raspberry pi wallet, since it's not a big deal.)
The reason for this strictness is that the EUDIW is intended for official or semi-official uses. For example, you can open a bank account with it, or use it as ID to get a mortgage. The bank must by law accept DOCUMENT, the state guarantees that DOCUMENT is correct, and you get better privacy than handling over a piece of plastic that is then photocopied by who knows whom. This is the tradeoff of the current EU law. It would be inappropriate for this kind of official, passport-like documents to store attributes such as your profession (journalist or whatever), and nobody is talking about it.