apache had not fallen first because netscaler is load balancing between 26 web servers and they are already pre-configured for all these heavy load.
Also - big question for me: were you using a ridiculously old version of mongo? I have yet to run into anyone who is on 2.0 or 1.8 and is still running master/slave (as opposed to replica sets). Sounds like your writes weren't working when you moved to replica sets because your client wasn't properly configured.