This is a breach of fiduciary duties. Maybe OP can get away with it for some time, but if the new venture goes well, it will be very ugly.
Better for all the worthwhile people to resign and start a new entity with no IP overlap with the old one. It still might get ugly down the road but OP will be in a much better position to defend.
If the CTO has really been that bad, then I don't see much downside to a fresh start.