Mainframes are exquisitely "balanced" machines that match processing capacity and IO for a relatively small set of workloads. You won't see mainframes doing scientific number crunching or AI training, but you'll increasingly see them adding more AI inference into their transaction processing workloads for things like fraud and anomaly detection. While their CPUs are prodigiously fast (the one announced is designed to run at 5.7 GHz, and the cores themselves are huge compared to other architectures), a lot of emphasis is placed on the IO systems that connect the machine to storage and other systems so that the CPU is busy all the time. It's very normal to run a mainframe at close to 100% CPU utilization.

Fujitsu replaced their SPARC CPUs with ARM, I think they were pretty good, but last I read about them was a few years ago.

It's not entirely clear what you're asking but... AGI, Vera, X2, Spark.

We have some older Power9 with NVIDIA V100. Nvidia drivers stopped a while ago (before V100 were considered “old” also for x86_64).

The main problem is software support by most machine learning libraries. While it is normal to have to compile many things from source when the binary is not provided even by some third party, in some cases the software will not compile on ppc64le and surely is not tested to work.

In 2019 the two fastest machines in the world, Summit & Sierra (see top500.org) used ppc64le + Nvidia GPUs.

Yes. There was a generation of POWER that offered NVLink.