If you feel like being social, this is a good excuse for striking up a conversation with someone nearby who seems interesting. Ask if the person can keep an eye on your stuff for a few minutes. Assuming he/she is friendly, after you get back and say thanks the ice will be broken and you can chat if you want to. I suppose it's still a bit riskier than taking your stuff with you, but seems minimal.
The trick I use is to pick someone that has line of site on my things, so they pay attention even if they don't mean to. And ideally to pick someone with a laptop, and preferably a better laptop than yours, that makes them a tad more trustworthy, because at least they probably won't take off with yours.
If the neighborhood is sketchy or there is no one around, I take it with me.
As for restroom cleanliness, worrying less about that is one of the many benefits of eliminating caffeine from my diet ;)