I was recently onboarded into a project which is using Python in a server-less setting, and when my colleague walked me through the concept of setting up a virtual environment for each one of my python projects I could not really believe it. It seems like you practically need to containerize a python code-base to make it repeatable.
While I get that you may have been frustrated with a Python setup, concluding that it's not suitable for "serious" production software isn't a proportionate response, and the evidence of many companies using it for "serious" production systems suggests that it is in fact fine. Examples include Instagram and Dropbox.
> It seems like you practically need to containerize a python code-base to make it repeatable.
This is pretty much the case for any interpreted language that has dependencies that aren't compiled into the binary. Arguably even for anything that doesn't produce completely static binaries.
Yes Python has a learning curve for production use, all languages and ecosystems do. In Python a fair bit of this happens to be on deployment, rather than in, say, compilation stages. Python is not out of the ordinary, it's not the worst. It's not the best either, it's fairly reasonable for its age, complexity, and typical uses.
It's not so much that I was frustrated by this, it's more the case that given the other ecosystems I have worked with, I was kind of shocked that people actually manage to ship projects with this type of tooling.
> This is pretty much the case for any interpreted language that has dependencies that aren't compiled into the binary
I don't agree. I think npm does an excellent job of making js packages repeatable for example.
> In Python a fair bit of this happens to be on deployment, rather than in, say, compilation stages.
To be honest, that's a big part of the reason I'm not a fan of interpreted languages in general in production environments. I think something like Python is fine for a setting like a notebook where you only have to worry about your own workstation, and you just want to iterate quickly without a ton of constraints on performance or robustness, but for running code in an un-supervised environment? I would much rather have a compiler checking every possible issue ahead of time rather than relying on testing and monitoring at runtime.
And Python seems particularly bad, because if I have to use a virtual environment to get the same code to run consistently on my workstation and my colleagues, that just gives me no confidence that I know how it's going to behave in some cloud execution environment. I'm not saying Python is "the worst" but basically all the properties of Python as a language ecosystem seem to run counter to what you want in order to produce robust, reliable production software.
Node I'd say is about as complicated. You still need to care about the Node version, about as much as the Python version. a node_modules directory is pretty similar to a virtualenv, it's not portable, maybe a little more reproducible by default, but no better than a pip-tools managed requirements file in my experience. Package upgrades are a little trickier than with pip-tools. You still need to care about things like what version of OpenSSL is on your machine, or do you have libpq installed, etc. One problem with JS is whether to compile and if so, how, what do you run, etc. You probably don't want to ship a node_modules to production with all the random package build scripts that it includes, so there's some additional complexity there introducing a build process.
The ergonomics might be better with npm but they both solve the problem of consistent execution across different machines pretty similarly.
Also npm is just one tool which behaves consistently and takes full responsibility of making a project portable. With Python it seems as if it's at the whim of the developer to choose from a variety of tools to complete this task.
In other words, if I see `pacakage.json` in my project directory, I know it will tell me everything I need to know to run the project. With Python, if the previous developer hasn't documented what tools they were using, I might have to do some archeology to understand how they have encapsulated the project's dependencies.
Or worse, if they never encapsulated their project's dependencies, I might just have to guess how the environment was set up on their workstation and try to recreate it through trial and error.
But the 'python version' problem (and really just any notion of a 'system python' in general) is still absolutely one of the bigger pain points of the ecosystem. I can say that for nix OS's pyenv ( https://github.com/pyenv/pyenv/ ) is a real godsend for this though. Pure-bash, no bootstrap problem, doesn't need to be 'installed' / can just be cloned and used, and has been able to reliably install any version of python I've needed into any of my relevant platforms in a well isolated ~/.pyenv/version/v3.x.y from which I can '~/.pyenv/versions/v3.x.y/bin/python -m venv .venv' in my project's dev bootstrap/Makefile. It's not part of my production-bound Dockerfiles which just base off of the official python images but it's invaluable for development and testing.
If you are going to use it I definitely recommend cloning it directly to ~/.pyenv rather than installing it via brew or somesuch as those tend to lag further behind the most up-to-date python versions than I'd like. And this is of course in no way a general solution to all of python's many packaging woes, and it'll probably never really be as simple as packages.json for many very valid if unfortunate reasons, but the 'python version' problem is at least pretty well handled by pyenv.
Versus, one of Python's bigger challenges with environment management is that you need to come up with an environment isolation mechanism that won't also interfere with the operation of the OS itself, or cut you off from being able to use userland tools that rely on it.
Which. . . Python being an essential part of virtually every production system out there being one of the root causes behind a situation that makes people feel skittish about its appropriateness for use in production is a fun thought. It's a particularly delightfully Unixy brand of ouroboros.
It seems crazy to me that the precise Python version isn't defined in the Python file itself, and the interpreter is responsible for sorting out compatibility issues.
C and C++ have a a fairly similar problem with dependency hell due to shared libraries being installed in a single system-wide location. They have their own hacks for getting dealing with it. I don't know which way of doing it is technically superior, all I know is none of them offer a developer experience that I'd choose for myself if I were designing a new programming language in this day and age.
FWIW, I give Python some credit for coming up with a solution that doesn't rely on containerization, and therefore will work consistently on many OSes, and not just Linux. Containerization is an option, too. It's really down to whether you need a system that works for one language on many OSes, or one that works for many languages on one OS.
Or, if you'd rather keep it really simple, there are zipapps, which are Python's answer to fat binaries. Similar to uberjars in Java. Or to good ol' static compilation in most the languages I like to use even when someone isn't paying me to do it.
(That said, there is one thing about environment management in Python that absolutely drives me up the wall: There are more than a fistful of competing tools for doing it, and a zillion different ways to work with them, and no clear preferred way. Which sucks for everyone, and makes learning the ropes without a good mentor way more of a chore than it needs to be. That's not too far off from C, either, though.)
Generally, newer projects are settling around Poetry.
Yeah I guess this is really my main issue. Like it seems like every Python project needs to come with a README to explain which tools are required, and maybe even which order they have to be applied to get the desired result. It just feels like an almost absurd step back when compared to other language ecosystems where these are well-solved problems.
As a user it's possible to put up with this, even enjoy it, when there are a handful of tools using it. A container running your webserver and its dependencies like database and caching for instance, that's reasonable.
Most serious production workflows are containerized these days -- are you sure that this supports your conclusion?
It's not that Python is a "bad language", but all these things taken together just basically work against everything I am trying to achieve in a solid production environment.
But there are a lot of other languages out there besides C++ and Python.
Not really, no. Only if the compiler, compiler version, compilation flags, kernel version, platform, platform version, cpu version, directory structure, shared object versions, locale, environment (LD_PRELOAD, LD_LIBRARY_PATH, TZ), config in pwd, .local, and /etc or maybe $XDG_*, "~/Library/Application Support/" are all the same.
I think they're referring to unpredictable behavior under future versions of language VMs and garbage collectors vs. statically compiled binaries with either no GC or a statically linked GC library.
Most of the time, all of that machinery going on behind the scenes is very convenient, but coding strictly to POSIX and the C/C++ standard libraries / STL and keeping your compilation artifacts gives you a lot more stability than coding to some version of the Java standard library (or other language VM) and keeping around your JVM installer with hopes that the installer will agree to install and run on future systems.
I remember being bitten by testing our product against the earliest JRE we supported and also the latest JRE sun had released at the time. I was aware that reading environment variables was deprecated in the earliest JRE I tested with, and I knew it had been un-deprecated in the latest JRE I tested, and the methods I was using were present in all of the JRE versions in between. What I had missed was that there was at least one JRE version in the middle where the deprecated environment variable reading methods actually threw runtime errors.
I've also run into issues in the past with the version of MSVC I was required to use having an installer that depended upon some ancient version of the CLR which would no longer install on my machine. There were some workarounds, but I needed to talk to several of our helpdesk people before I found someone who knew the required magic incantations. Had the MSVC installer been coded strictly to C++ stdlib, STL, Win32 and MFC, it would have run just fine on my modern desktop.