That being said, the virus isn’t magically going to fill the entire space equally from one infected person. There is a distance factor involved, and the closer you are unmasked the higher your risk. So only unmasking at your table cuts your risk pool down to the tables immediately proximate to you, rather than everyone you walked by, and anyone you shared the bathroom with. It’s not as safe as staying home, but it’s better than being unmasked the entire time.
Oh, and the risk isn’t going to hang around hours after the fact, unless if you’re in a completely unventilated space. Commercial restaurants change their air over a recommended 8 times an hour.