An argument that I haven't seen yet is that some of these providers allow to choose the mail address used, allowing anybody who knows or can guess the address used to take over the account.