Not really. Not under OSX, brew makes this simple. Under Linux, worst case, you just download the source and compile it. (Not sure about Windows though.)
And pyenv is a wrapper around “downloading the source and compiling it”.
Brew makes this easy? Are you sure? I installed python 3.9 with brew and it’s magically become a 3.10 install by upgrading another package
The version pinning on brew certainly isn't great, but you can do: