I use Debian's system Python 3.10 install for most of my stuff and it works really well for me. Some things I install via pip but the key libraries (e.g. PyTorch) from source.
I wonder if our definition of "non-trivial" might differ if neither of these apply.