100%. I stopped writing anything that had to be deployed (basically everything except Jupyter notebooks for data stuff) in Python because it’s truly a nightmare. Go and goreleaser is great for writing a CLI (and if it’s public, it can auto generate binaries and upload to GitHub, create a Homebrew/Scoop bucket, etc)
If I have to do weird things with data structures or complex logic, I reach for Python, but I do it one of two ways: 1) so stupid that "python foo.py" works (no external deps), or 2) publish it to pypi so the user can just 'pip install foobar'.
What I personally do is write POSIX shell, then add in things like arrays, $BASH_SOURCE, printf %q, and a few other handy features that have been around for a while. To test it, there is an official Docker image for Bash v3 (https://hub.docker.com/_/bash).
But! Almost nobody gets it. If you're into operations: "here we write our tool chain in Python". If you're into dev: "here we write our tool chain in js/go/php/etc/etc Bash would just be a +1"
+1??? You still need to write bash, or shell in general, for orchestration of build/ci/cd pipelines. If someone thinks that in 2021 they can escape learning shell, they are out of their minds. So no, it's not a +1.
Bash is the language that is not only available quite everywhere but it's also the one that allows you to write oneliner solutions https://twitter.com/andreineculau/status/1466107915450916867... and move on.
As devs we should spend 10x more on solid foundations that work as many of us, for as many years to come, than we do on hip-new-thingie-that-promises-to-fix-all-my-problems. It's as if we took the regex joke and our brains could only comprehend: I'm never using a regex. Keep it simple? Never.
Heck no. Bash is a terrible language by modern standards, with obscure syntax, many weird quirks, no type checking, no return values... How could developers program in Bash before Shellcheck!?
I think that internally, I draw the line at around 100 lines. Even refactoring a program shorter than that it's not a pleasant experience (not difficult, but requires a great deal of attention, when compared to any modern language).
You're right about them needing to be short. Ideally every program should be short. But scripts are especially supposed to be short, because they're never supposed to get complex (or be fast/efficient); that's when you need a "real" program. Even the bottom of the Bash man page says about itself: "BUGS: It's too big and too slow."
(fwiw, back in the day we didn't use linters, we just had to write it all correctly the first time. punch card typos are a memorable lesson in checking your work)
The fact that every language has a variable amount of a certain property doesn't mean that the amount is the same for all the languages. The amount of domain knowledge required for simple programs is considerably higher in Bash, than any related language; even iterating the lines of a string can be unintuitive to casual users.
Bash is weakly typed, and weak typing is considerably unsafe (bug-prone) than any other related language (e.g. Python, which is has taken has share of shell scripting in modern operating systems).
Returning values via stdout is a workaround. If other parts of the function send messages to stdout, one needs to use a combination of redirects, which the vast majority of the programmers don't know. Calling the ability to return a value from 0 to 255 support for return value is a big stretch.
I think that Bash is a necessity, and I frequently use it (and it's fun-ish for small scripts), but it definitely doesn't fit the definition of "easy", being considerably more error-prone and unintuitive to any comparable language, possibly even when compare to Perl.
Using the zipapp module, it is possible to create self-contained Python programs, which can be distributed to end users who only need to have a suitable version of Python installed on their system. The key to doing this is to bundle all of the application’s dependencies into the archive, along with the application code
Venv+pip or pipx are also fairly good if you have a pip registry.
And there's the problem
From my personal experience, there's so many things that can crop up, not limited to:
- Conflicts with system packages from e.g. apt
- Conflicts with system packages from easy_install/pip
- Mismatching Python versions (even among 3.x with syntax additions only available in newer versions)
- Different dependency management systems (pip's requirements.txt, setuptools's setup.py, poetry's pyproject.toml, etc.)
- ... and the list goes on if you start talking about C extensions!
Additionally, not everyone does things in the same way which means the easiest way to package Python tools tends to be just bundling Python + its entire standard library together. Contrast this with Go, where `go install` does basically the right thing 9 times out of 10. I'm not a huge Go fan, but the convenience of `go install` is really unmatched.
I mean if you upgrade from x11 to wazland and zour app depends on x11 then sure, but I still use binaries that were compiled in 90s (some because I dont have the source)
And windows is also very backwards compatible.
Python is hell for the latter. I support Windows and Linux, so I can bend pyinstaller to my will with enough effort, but I need to have a special Windows VM to "cross-compile".
For the former, though, it's a matter of creating a setup.py and telling the user to pip install the repo on GitHub. Or you can use Poetry, which makes things slightly easier.
There's some python magic for determining the version, but if you use a feature not supported by the version it raises an uncatchable exception...because python's parser has control before your code has control.
This isn't the only deployment issue, but a good example of what you're in for.
In short a package manager + git is the easiest way to distribute and maintain software ... once you know how to use them. Configuring in a new environment? There is nothing that works reliably, other than maybe a full blown human level intelligent agent that is a senior developer with direct access to poke the system until it works.
Further, developers with little time or experience packaging software for regular users may balk at what they rightly perceive as a route that is more difficult and creates more work in the long run because as a dev you can't just drop a "fix pushed" message your messenger of choice and go back to work.
Static linking seems like it is a solution until you have to distribute to multiple operating systems. Only things like cosmopolitan libc have been able to overcome this, but even that requires a binary blob.
In long ....
I maintain a dozen or so python packages. Small stuff, mostly used by myself and my research group. At some point I needed to make some of that functionality available for other devs and some non-technical users. It was and is a massive pain.
Why was it such an issue? I think it is because many of the quick internal scripts were originally written assuming that only technical users as advanced or nearly as advanced as the author would be using them, and that any such user would be able to use git, configure their environment, and generally keep their system up to date using a package manager.
Aside. Add to that the fact that my dev environment is Gentoo, which has one of the sanest python environment management solutions available, and every time I step outside it or get reports from the poor lost souls trying to run my code "elsewhere" I enter a world of sanity destroying chaos. The workarounds that I created to try and mitigate some of the issues were poorly designed because I tried to YAGNI without realizing that you inevitably gonna need it (YIGNI?) and that the technical debt surrounding issues with interfacing with the environment is 10x more time consuming than anything else, especially if it is in the form of reports from souls lost in the warp.
Even if a dev can use git and a package manager they can still get stuck configuring the environment. Seems easy until there are secrets to distribute or someone needs to run something in a sightly different environment and nothing works.
For non-technical users it can be a stretch to get docker set up correctly, especially if they need X11. If I can get ssh access to the machine then there is a chance, but that doesn't scale beyond maybe 2 or 3 users. I have seen enough of this to have started to seriously consider just learning how to create installers for macos and windows and build them via CI. The long term solution is to switch off python completely, but the legacy code will continue to haunt us for a long time.
I've learned docker to try to create controlled environments. I've looked at statically linking SBCL to musl libc. I've looked into cosmopolitan libc. I've explored trying to use Emacs, elisp, and org-mode as another possible route to bootstrap stable and predictable environments. I've looked into Gentoo prefix. I've wondered whether I could get users to install Pharo, but realized that it is already a tossup whether they can get docker running. I'm so sick of having to repeatedly do searches in high dimensional configuration space (fight with the environment) just to get some random code to run for a user who can't deal with the configuration space themselves (and it's not their fault!). At this point it would be easier to wire all my users up to a mitogen botnet ... except that some of them use windows!
Telling a user "install python" or "install x" and then run this code leaves out the "and resolve the 10 completely novel errors you will encounter along the way." This is because there are at least 5 ways to "install python" (oh, don't forget the even bigger nightmare that is "install pip") on any given operating system (forget distros [0]), and each one results in a setup where things like data or configuration files are installed in different places with no obvious way to figure out where those places are, or you are forced to resort to a runtime lookup that causes CLI program startup time to explode to hundreds of milliseconds, making the entire point of having a quick CLI script moot.
Hello world may be the easiest program to write but it is by far the hardest program to run. Dependencies? Data? Network? Good luck.
Based on the idea choose the right tool for the job it seems python is not right for this job.
An alternative would be to write in Perl5, perl is pretty much fixed, these days (if you insist on not using containers)
Try to compile your go tools in year, or two. If think that you will be hard pressed to find the dependent library versions. Packaging is not the strongest part of go. A version in your package translates to a tag in the referenced git repository, now sometimes people do funny things with their repositories, and older tags disappear.
Python and Perl5 are better, at packaging. At least you have all of the older library versions on https://pypi.org/ I think, that the decentralized approach to packaging (as used in golang) is far from optimal.
Maybe this is all due to the fact, that google is using a mono repository, internally for their own code. So that they are not eating their own 'dog food'.
Sorry for ranting, judgements like 'right tools' really get me going..
I think GP was just saying that most times, fewer moving parts is better. Avoiding Docker is great if you don’t need it (especially for a CLI tool). Obviously there are trade offs that must be evaluated in the context of a given problem and domain.
you may be, but the point is that the time you save writing in python is paid multiple times when each person who wants to use your code struggles to install it and get it working.
Mainly because volumes are mounted as the host user UID, while the container might be running any other user, often root.
Quite baffling they haven’t made this easier since it would be a real killer feature unlocking even more potential from containers.
Another use case is running a http server locally, for something that has a graphic ui. Like one of my projects here: https://github.com/mosermichael/s9k
Just keep them up to date with changes and always use system provided packages, and there is no issue.