Unless they were somehow able to keep the pilots completely unaware of what's going on, that wouldn't work. It's a pretty extreme response, and would risk passengers (assuming they're still alive), but depressurizing the cabin would pretty quickly stop any attempts to break into the cockpit.
Even if attackers could get to supplemental oxygen in time, those little bottles don't last forever, and there'd be an opportunity for crew and passengers to regain control.
I think there are other last-resort actions pilots could take to thwart a takeover, like engaging in extreme attitude/altitude changes in order to knock out the attackers.
Remember, we're operating under the assumption that if the hijackers win, everyone on the plane is likely dead anyway, so putting passengers at risk to stop them should be considered acceptable. The only way attackers can realistically win is if a pilot is duped into letting them into the cockpit. Not impossible, but that reduces the possibilities quite a lot.