You're not a real health nut unless you're activating your almonds
Some nuts will have trace amounts of mold on them, depending on how they're sourced or how long they're sitting around. I've read that it's most common on pistachios. Not sure to what extent it actually happens but rinsing the nuts certainly won't do any harm.
Buy some walnuts. Try rinsing them before eating, to wash away the walnut dust. You may find you prefer the taste after washing.