Many smart devs tried to help me, and I added some of their recommendations:
- caching views to a bitmap using Metal
- trying smaller windows so that less pixels have to be redrawn
Unfortunately nothing made a noticeable difference, this will have to wait until I get a really boring weekend.