but all transactions are public [1] Thus once you inform gov which are your bitcoin wallets, they can look through the block chain. Ironically, this means they will have much much more info on you than if they just had your bank account number.
[1] https://en.bitcoin.it/wiki/Transactions