Use Homebrew to install pyenv and pyenv-virtualenv. Then use those to install pythons and collections of packages. You'll have stable development environments.
If you're having trouble installing something with brew on M1, run arch x86_64 brew instead of brew to do a Rosetta installation.
https://github.com/pyenv/pyenv/issues/1643#issuecomment-7293...
For installing Python on the M1 Apple Silicon. This will install x86_64 versions, but those will work just fine.
Other replies here appear to assume basic compilation issues, but the issues I've seen on various GitHub projects are mostly related to these more insidious C toolchain challenges. They're not M1-specific, but they are manifesting on M1 machines.
My current Mac is going to be my last. Linux on Thinkpad is next