Then they could be sued by a minority shareholder for knowing they should take action, but then failing to, and resigning instead. Failing to take an action is still breach of fiduciary duty.
Only if you believe it is their fiduciary duty to remove him, and that is a long stretch unlikely to make it though the lawsuits.