I have spent a ridiculous time building this so I'll take the opportunity and share.
It builds python wheel packages in a build container and installs them in an app container. Works great for cpython and pypy. Also allows to build for alpine and works for most other languages. We started to build basically everything that way.
https://gist.github.com/tuco86/67d84dfb27268b1faf05d2dbb1acb...
Ok, I kind of cheated and added the user just now. Sue me. Also posted this in the other Docker related news. Sue me again.