I'd expect that the name and address will still be stored in their payment processor.

Then the invoice itself (containing your name/address/payment method) will need to be stored by the company in some form for accounting regulations.