If I were in that situation, I would do one of a few things:
1. Donate more to charities that I respect because they are better prepared to help than I am. Furthermore, if possible, volunteer in some way in your local community. What you do for volunteer work depends on the local community's needs. The main thing with this is to do things regularly, but it doesn't actually have to be fundraising. Bonus points if you can use your professional skills to help somewhere out - but if not, no worries, there is usually plenty to do.
2. Get involved in politics, up to and including running for office. Make sure folks have actual choice on the local ballot for all positions. Recruit like-minded folks if you can, especially for those positions that only one candidate is running (in total). You can even volunteer to work the polls and/or making sure disadvantaged folks can get to the polling places - among other imaginative things. This provides some catalyst for longer-term change.
The option with a non-profit job is completely up to you. In the end, it will still be a job, so I'd take care to make sure you'll enjoy the work (as much as possible). I'd likely keep my eyes open to see if the right opportunity comes about.