The Typescript transpiler does definitely slow things down but I've tried the "only check types during build" approach and while it starts out very nice and quickly, you end up wasting all that time you would've waited on tsc by chasing down impossible to find typing bugs and doing the necessary code restructuring to fix the problems your IDE missed. Only finding out about these bugs when pushing code (or even worse, when opening a pull/merge request) can be a lot more annoying than giving tsc half a second to catch up.
I do wish that tsc was faster, but on the other hand I can't say it's been that much of a problem for me.
Perhaps this is because I'm almost exclusively on Linux. tsc operates on tons of tiny files and Windows is notoriously slow at dealing with that, even with AV exclusions.