Thanks for the suggestion. Adding these flags halves the startup time:

  $ time ./pwsh -NoProfile -NoLogo <<<'' >/dev/null

  real 0m0.456s
  user 0m0.388s
  sys 0m0.040s

Much better, but I still wouldn't use this even as an interactive shell, as I open a few hundred terminal windows each day.

There's no defender (or anything similar) on that other machine. It also has an SSD. With an older HDD powershell started in about 10-15 seconds there (while cold startup time for bash was something like a second, and instantly afterwards).