What speed are you looking for in your CI pipline that you are not getting from mypy? Perhaps you are working on a much larger codebase than I am but the app I use it in takes under a minute. And all the unit tests take longer to run.
A 25k LOC Django project - takes 15-20 minutes to run mypy... Unit tests take about 3 minutes.
That's locally on a fast (ish) laptop. GH CI takes 10 minutes to run tests, so I'm a little hesitant to see how long mypy takes to run.