Being from JAVA Shop for long time, If I have to switch between different version of JAVA, all I do is change JAVA_HOME to point to correct version, go to base project directory and "mvn clean install" does the job. :).
The one thing I thought was neat here was pipx. I do have a few CLIs set up in my default conda env and haven't run into any dependency problems yet, but have occasionally tried to use them while another env is activated. Having a separate env automatically created for the entry points is a nice value add.
Need to have multiple versions of python installed and easily accessible? Use pyenv.
Need to run tests across multiple versions of python? Use tox.
Need to freeze environments for deployment purposes? Use pip-tools.
Need to freeze the entire operating system? Use docker or vagrant.
Don't use tools you don't need. That would be silly.
However, you can run into issues if you are using different versions of Python, or Python on different operating systems.