The performance isn't really that bad but it's definitely a hit. The bigger issue is that it causes apps and services and occasionally also the kernel to throw MTE SIGSEGV which becomes a substantial problem if the developers for said apps or services don't care enough to fix them.

So the TLDR is that the main issue is that it causes a whole bunch of stuff to segfault when bad memory accesses occur but where they wouldn't segfault without MTE. It makes the phone feel unstable to average users since stuff just crashes with no real end-user-facing explanations.