Then I point Pycharm to use that as my remote interpreter through docker compose.
Not watertight, but a hundred times better dev experience.
Then I point Pycharm to use that as my remote interpreter through docker compose.
Not watertight, but a hundred times better dev experience.
The flaws of python's packaging system (as well as the GIL and other hangups) emerge from the tradeoffs of inventing a language optimised for reusing and connecting as many disparate system binaries as possible.
Its not surprising that such a server scripting language is tightly coupled with the server environment that it runs in. Docker is just one way to ensure a reproducible server environment across machines.
What do you think it could have done differently?
- Make dependency resolution deterministic by default. Unfortunately the whole ecosystem has to buy into it, but the benefits are huge.
- Stop building castles of sand by making more layers of tooling that have to run in Python (and will therefore run in some random poorly managed Python). The language install can't manage the build system install - if anything the build system install should manage the language install. Adding pip to the language distribution was such a backwards decision that it marks the point where I gave up on Python ever fixing their stuff.
- Ignore Linux system package managers (apt etc.), they have a fundamentally broken model and will infect your language ecosystem with that breakage if you try to cater to them.
These are language agnostic and can get the job done.
>>
>> These are language agnostic and can get the job done.
> Because I have dozens of colleagues, each with their own os/distribution, developing an application that needs to lock down dot releases of dependencies to the versions running in production
I assume that the production environment isn't running dozens of os versions and distributions. For development, using a VM or container running the same os version and distro that's used in production and using that OS's package format for packaging the software and installing it in the dev environment (on the container or VM) would work. You're testing to see if the software works in the production environment, not someone's preferred os/distribution.
> and update them at the same time
I'm not as familiar with apt, but dnf has a version lock feature that would allow you to lock down the dependencies to specific versions. You could update them and test them during development and update the version lock file to pull down the updated version of the dependency when updating production.
This is something that's typically handled by the package maintainers for each linux distro, rather than the ones who developed the application. Some developers maintain their own public repositories for packages they built and instruct end users to add their repositories to their package manager config, but they typically will also include the source archive (if it's open source) and build instructions for those running distributions they haven't built packages for.
Looking at Virtualbox[1], for example, in addition to Mac and Windows, they built packages for Redhat, Ubuntu, Debian, OpenSUSE, and Fedora, and they provided the sources along with build instructions[2] for those who are running distributions where there's no pre-built package. In fact, the last option is the most flexible one though it requires a bit more work for the end user.
And virtualbox is actually a good exemple of what I'm saying: despite being one of the major software in one of the field where Linux is strong if not dominant.
- they feel obliged to distribute themselves their Linux packages - they have to distribute 1 package for windows, 2 for macOS but 12 for Linux.
But this also brings up the issue of vetting dependencies. If you're pulling in a dependency that pulls in 10s of other dependencies (direct and indirect), it gets difficult to vet them. PyPi and npm have already had issues with malicious packages being uploaded. On the other hand, I haven't really found the large number of dependencies being an issue for python packages available in the OS package repositories, and I'm not aware of any incidents with those repositories unlike PyPi and npm.
You mean a link to a malformed .deb packages?
It's not difficult to do but it's also the wrong way to do it.
So in practice we're back to shipping whole distributions using Docker (or, like we did, abandon Python as an option for development but keep .deb packages for deployment).
Additionally, even if you only rely on a container - still no: other container engines have a lower attack surface.