While this is true, at least in the early pandemic when social circles were pretty tight (often just your household, but even if not quite a bit smaller than during normal times), the virus has to be introduced to that social circle, it won't just manifest itself out of nothing in one of it's members. Masking when not interacting with that circle will help keep the virus out of that circle of people in the first place.
This is a very hard thing to maintain for long periods of time, but in the context of 2020 it's a reasonable approach to take.