Stealth bombers aren't going to dodge missiles. They need to reach their target before they are noticed at all. Otherwise the enemy will mount a response and intercept the bomber.
It's sort of like if you're deer hunting and spot one through binoculars a mile from you moving away. You're going to have to get really lucky to take that deer.
Given that there are very few B2s, they're only going to be sent against the most valuable and well protected targets. Those are the most likely to have very alert defensive assets in exactly the right place to go after the B2s.