I think we have a tendency to want to use our Special Talents to help people, when that's not always the most effective thing to do. For example, I prefer building websites for non-profits any day of the week to running anti-virus, but that's often not nearly as helpful as just upgrading everyone's copy of Internet Explorer and removing all the toolbars (speaking from first hand experience here; I've done both). Also, if you only have a day or two to offer, you need to be realistic about what you can finish. No one benefits from work half-done, and everything needs maintenance.
So instead of waiting for the perfect storm of Your Skills meets The Perfect Charity, just go down to the office of a charity you care about and offer to help with IT stuff. Its not glamorous, and you might not self-identify as an IT person, but you'll be more able to help than you might realize.