I was in this exact situation together with many colleagues. Just leave. It sucks that you have to leave what you've invested, but it's a fallacy to think that it will get better. The project manager needs to be punished and you need to find a better job.

Take it slowly and rationally though. Do not give them any hint that you are leaving at all and start a slow powerful search for your next adventure. My number one tip? Make sure you get a good manager which is both good to you as well as in control of their immediate environment meaning so that they make sure you can keep your sanity as well.