The benefit of masks is primarily that it reduces droplet transmission from carriers, many of whom are asymptomatic.

You can't propagate a droplet to your face by touching a surface and then your mask, if it never lands on that surface to begin with.

I'd really like to do away with this misconception: masks are primarily to prevent transmission, and only secondarily (and it's a distant second) to prevent infection.