Ruby: A great language for shell scripts
lucasoshiro.github.io
lucasoshiro.github.io
Also ruby is great in allowing complexity to grow smoothly, no sudden hiccups. You start with just one line (everything goes into module main implicitly), extend it to a single-file script, require some built-in libraries, then add a module or helper class in the same file, and only then maybe extract those files to required files, add gems, whatever. No boilerplate whatsoever, no jumps, no big rewrites.
meanwhile, a lot of tooling nowadays is written in Go, and I have no idea why, it's not friendly for os manipulation at all, and number crunching power is not needed in many, many tasks of that sort.
I. E. ‘sed -i’ is only in GNU sed. Same with ‘grep -P’.
Otherwise nobody thinks of it because most likely it is not being distributed.
All of those have wildly different behavior depending on their "flavors" (GNU vs Busybox vs BSD) and almost all of them depend on libc being installed.
https://github.com/oracle/truffleruby
Unlike GraalVM Java, as far as I can tell TruffleRuby doesn't provide a bundler that can create a single executable out of everything, but in principle I don't see why it couldn't.
https://www.graalvm.org/latest/reference-manual/python/stand...
I'm not sure I'd try replacing shell scripts with natively compiled Python binaries. That said, I use a Kotlin Scripting based bash replacement in my own work that has many useful features for shell scripting and is generally much more pleasant. You have to "install" it in the sense of having it extracted somewhere, but it runs on Win/Mac/Linux and can be used without root etc.
Even with a good type system, a trimmer/linker has to be enlightened of many special idioms and patterns and perform flow analysis, and in the case of dynamically typed languages or languages with reflection - to analyze reachability of reflectable members and trim otherwise spaceous metadata. It took significant amount of work in .NET's ILLink in order for it to be able to produce as compact binaries/libraries as it does today with .NET's AOT build target, and it still required a degree of metdata compression and dehydration of pointer-rich data structures (something which, funnily enough, Go doesn't do, resulting in worse binary size).
like deno or bun, but for Ruby
artichoke ruby is the closest we've got
Simpler than having to worry about Python versions, let alone dependencies.
It works well but with one huge caveat: although you bring the stuff required to reconstitute the venv with you, you’re actually still using the system’s python executable and stdlib!! So for example if you want to make a project targeting all supported Ubuntu LTS versions, you have to include the wheels for every possible python version you might hit.
Ultimately this boils down to there not really being a story for statically compiled python, so in most normal cases you end up wanting a chroot and at that point you’re in a container anyway.
There are other options I didnt look too much into, e.g. Beeware
The way inline script metadata works is that your script declares arbitrary dependencies in a structured top comment, and a compliant script runner must provide them. Here is an example from a real script:
#! /usr/bin/env -S pipx run
# /// script
# dependencies = [
# "click==8.*",
# "Jinja2==3.*",
# "tomli==2.*",
# ]
# requires-python = ">=3.8"
# ///
For really arbitrary hosts with no guarantee of recent pipx, there is https://pip.wtf and my venv version https://github.com/dbohdan/pip-wtenv. Personally, I'd go with `pipx run` instead whenever possible.
[1] I recommend shiv over PEX for pure-Python dependencies because shiv builds faster. Have a look at https://shiv.readthedocs.io/en/stable/history.html.
I'm thinking of trying out Mojo in large part because they say they're aiming for Python compatibility, and they produce single-file executables.
Previous to that I was using PyInstaller but it was always a little fragile (I had to run the build script a couple of times before it would successfully complete).
Currently I'm using pipx and Poetry, which seems pretty good (100% success rate on builds, and when my 5-line build script fails it's because of an actual error on my part).
Which is a round-about way of asking everyone:
Does anyone have any other good way(s) to build single-file executables with Python?
[1]: https://nuitka.net/
From the website, "D's blazingly fast compilation allows it to be used as a high level, productive scripting language, but with the advantages of static type checking" [3].
[1]Why I use the D programming language for scripting (2021):
https://news.ycombinator.com/item?id=36928485
[2]Adding ANSI C11 C compiler to D so it can import and compile C files directly:
It can do the Python venv stuff behind the scenes for you and it just looks like a single Python file.
I’ve used rust for this task but people get mad that I’m calling it a “script”. “That’s not a script that’s a program” which…sure. But so maybe we need another term for it? “Production-scripts” or something.
My experience is rewriting Ruby and bash buildpacks for the open spec CNCF Cloud Native Buildpack project (CNB) https://github.com/heroku/buildpacks
I agree that Ruby is easier to start and grow complexity, that would be a good place to start.
Most of them were so old that I would have had to skip like 3 generations of package managers to get to the one that's used this year (dunno about next year) if I wanted to upgrade or add dependencies.
With Go I can just develop on my own computer, (cross)compile and scp to the destination and it'll keep working.
Other than that, OS scripting is done in traditional UNIX tools, or Powershell.
In fact one of the reasons I rage quit megacorp for a second time was that I was required to use an Enterprise Chef instance that would log people out at random every 0-3600 seconds. I could throw plenty of deserved shade at my coworkers but Opscode didn't understand their product any better and I wasted more than enough time on conference calls with them.
A secondary reason is that Ruby has been very slow for much of its life, which means that for situations where you need to run a huge stack of scripts -- init systems, for example -- it would be punishing.
Ruby does have a terse and intuitive syntax that would make for a good system shell. Although it has some magic, it is less magical and confusing than shell itself. Ruby provides many basic data types that experience has proven are useful for shell scripting -- like arrays and dictionaries -- and they are integrated in a much cleaner and clearer way than they are integrated into widely used shells like Bash.
System tools that are written in Go may still make sense to write in Go, though. Go, it is true, does not have a nice terse syntax for short scripts and one liners; and it doesn't have a default execution model where everything is in main and so on; but that is because it is not a scripting language. Other languages used to write system tools and system services -- like C, C++, Java and Rust -- don't have those things either.
This seems contrary to my experience. We took a large project from 1.8 to 1.9 to 2.0 to 3.0, and it was much easier than we expected. It was a lot easier than our Python 2 to 3 conversations were.
Python's is (present tense very much intended) notoriously one of the worst-managed transitions in programming language history, so that's not exactly a ringing endorsement.
The Ruby 2.0 migration wasn’t that interesting from a compatibility perspective; it certainly wasn’t anything like Python 2 -> 3.
And Ruby is __not__ slow compared to bash. I don’t where these myths get started, but someone needs to justify the Ruby-is-slow thing with actual data.
As an outside observer of the Ruby world, I have an impression that it was Ruby MRI that was slow. CPU-bound synthetic benchmarks like the much-criticized Benchmarks Game showed Ruby ≤ 1.8 a good deal slower than CPython 2. Here is an illustrative comment from that time: https://news.ycombinator.com/item?id=253310. People also complained about early Rails, and the perception of Ruby's performance got mixed up with that of Rails.
Then YARV came out, and Ruby became several times faster than its MRI former self on different benchmarks (https://en.wikipedia.org/wiki/YARV#Performance). With YARV, Ruby gradually caught up to "fast" interpreted languages. Now interpreted Ruby seems as fast as CPython 3 or faster in synthetic benchmarks (for example, https://github.com/kostya/benchmarks/blob/7bf440499e2b1e81fb...), though still behind the fastest mainstream interpreters like Lua's. Ruby is even faster with YJIT.
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
"There are only two kinds of languages: the ones people complain about and the ones nobody uses."
> showed Ruby ≤ 1.8 a good deal slower than CPython 2
It's taken me a couple of days to remember web.archive.org :(
CPython 2.5 vs ruby 1.8.6 (2007-03-13)
https://web.archive.org/web/20070219190706/http://shootout.a...
I learned enough PowerShell to be comfortable using it, and then picked up Bash and Ruby a few years later.
I longed for a Ruby shell for a couple years.
Ruby never had US market penetrative as perl or python, which were basically invented in the US, and congregated people from the academic realm. These things aren't decided based on meritocracy (no things ever are).
Python 2-to-3 was mainly worse than Ruby 1.8 to 1.9 because Python had already won, and had a much bigger and more diverse ecosystem.
Ruby 1.8 to 1.9 migration was by contrast way milder. It took 6 or 7 patch releases and around five years to release 1.9.3, the first from the 1.9 series people actually considered stable, but after that the community migrated because it was *significantly* faster than 1.8 . Python 3 on the other hand was slower overall than python 3 at least until 3.6. The fact that the community stuck with python through it all does say a lot about human psychology and sunken cost syndrome.
The moment you start asking the user to install things, you’ve opened up the possibility for writing a program rather than a shell script. The lifecycle of a piece of software is almost always one of growing responsibility. This cycle is devastating when it happens to shell scripts. What was once a simple script slowly becomes creaking mass of untestable, poorly understood code playing in the traffic of swimming environments (which grep you got, buddy?).
I guess I’m saying that once you open up the possibility of writing a program, you generally take that option and are usually happier for it. In the “write a program” world, ruby is still good, but it becomes a far harder question to answer whether ruby is still the right choice. There are a lot of languages with a lot of features engineers like.
My scripting language is bash in at least 99% of cases. I used to program in Perl when I need some complex logic. I stopped using it some 10 or 15 years ago when I switched to Ruby for two reasons: I became more familiar with it than with Perl and it's easier to manage data structures whenever I need something complex or classes. That doesn't happen often in scripts but as I wrote, I use bash for all the normal stuff.
I use Python for the scripts that start an HTTP server because it has the http.server module in the standard lib and it's very simple to write handlers for GET, POST and all the other HTTP verbs. The last example was a script to test callbacks from an API. I just implemented two POST and PUT methods that print the request data and return 200 and a {} JSON. I think that to do the same in Ruby I would need to install the webrick gem.
You can happily copy the zip of your scripts and all deps in the server.
You still do have to mind your versions, as always with python.
With a big difference -- Perl and Python will always be installed on these machines, whereas Ruby might need two deployment steps: (1) copy file, (2) install Ruby!
However I haven't worked at a company in years that gives anyone access to root anywhere except your own local machine or maybe in rare cases a dev box that is destroyed and rebuilt at will.
True, not true for Ruby, but with Golang and Rust you have an almost-no-dependencies final binary so the argument there does not apply.
> which grep you got, buddy?
For dev machines it's not such a tall order to require `rg` be installed these days.
Or when you have to start using all the combinations of characters to achieve f.ex. proper iteration through an array without word splitting. Etc. to infinity.
I've danced this dance hundreds of times and got sick of it. Gradually moving away from scripts and to Golang programs and so far it has been an improvement in almost every way, I'd say easily in 90% of the cases.
sure if it's your "script" and you know exactly the code corresponding to it it might be ok, but that's a lot of overhead for a disposable script.
Simple, ruby is not installed by default. Even Python, while it is on (almost?) all modern Linux distributions, is not installed on the BSDs.
Installing python and other general purpose tools gives any attacker that gets into a docker container many more tools to work with for getting out.
For docker, the trend isn't "build a general purpose machine" but rather "what can we slim this down to that only has the bare minimum in it?" This can be taken all the way to the distroless images ( https://github.com/GoogleContainerTools/distroless ) and means that the security team won't be asking you to fix that CVE that's in Python that you don't use.
If, however, you do need python in an image because that image's purpose is to do some python, then you can pull a python image that has the proper release.
Alpine for example doesn't ship Bash. Mac OS ships Ruby and its Bash is quite old.
The only tooling I know that’s written in Go is Docker.
I can easily provide precompiled packages for all sane combinations and users can just download one executable, edit the config file and be running.
Instead of having to mess with virtual environments and keeping them updated (they tend to break every time you upgrade the system python version).
CPAN is the killer feature of Perl. It just works. First off, most of the time I don't need a CPAN module for doing shell scripting in perl. Perl itself is rich enough with the file manipulations that are needed for any script of less than 100 lines.
My experiences with Ruby and installing gems have been less pleasant. Different implementations of Ruby. Gems that don't compile / work on certain architectures. Breaking changes going forward where a script that was written 2 years ago doesn't work anymore. Sometimes it's someone was doing something clever in the language that doesn't work anymore. Other times its some gem got updated and can't be used that way anymore. ... which brings us to ...
I believe that Go's advantages come into play when the program gets more complex that that 100 line size and it becomes a "program" rather than a "script" that has complexity to deal with. Furthermore, executables built in Go are most often statically linked which means that someone upgrading the libraries doesn't break what is already working.
The reason we don't see Ruby used more for shell stuff is because Python won this particular war. It's already installed on basically every Linux distribution out there, and this simple fact outweighs all other language considerations for probably >95% of people who are writing shell scripts in something that isn't Bash.
Personally, I don't much like Python, and even though Ruby is not my favorite language either, I find it much better than Python for this kind of work. But I don't get to decide how Debian builds their infrastructure, so in the end, I tend to use Python.
I don't really get this whole defaults being a blocker for tools choice.
We can either deliver a single executable (Go) or a Python script, as python is preinstalled on their distro.
If we'd want to use Ruby, it'd be a huge hassle of re-certifying crap and bureauracy and approvals and in that time we'd have the Python solution already running.
Python and bash are used in the real world most often because convincing your sysadmin/infra/boss guy to install ruby for one script is a hard sell when you already have good-enough tools built into the system that don't add risk/complexity.
PEP 668 pretty much negates this though. To do anything you need a python environment set up per script/project w/e
You don't need it if it's just a python script with stdlib, just like with raw perl.
Idk why people are pretending there aren't tons of useful libraries out there. Like if you want to script anything with aws, use yaml
There are useful libraries, I’m not saying there aren’t. I just dislike it when people include one as a dependency when they really didn’t need it.
Way too confusing, compared to go for example. Or hell, even Java/Kotlin when you use an IDE and it autoconfigures most things.
The BSDs do not have this problem (yet!). I hope they stay sane and keep using Perl/sh.
The best piece of code that I worked on was an ETL in pure Ruby. Everything in modules, simply to read, no crazy abstractions, strange things like __main__, abstract clssses or whatever.
Maybe others can chime in, but the main difference that is found in ruby developers is that they really have fun with the language making everything with a higher lever of software craftsmanship that other folks in the data space, e.g. Python of Julia.
What? Go is used because distributing a static binary without any dependencies is way better than asking each and every user to download an interpreter + libraries.
It's only relatively recently that I could really expect that the target system would have python3, and then I'd also have to deal with some really annoying errors (like python3 barfing on non-ASCII comments when reading a source file with "C" locale, something that used to work with python2 IIRC, and definitely was an issue with "works on my machine" devs).
venvs are horrible, even compared to bundler.
But the python2 era left imprint on many who think it's just going to be there and work fine.
No-dependencies final static binary.
> it's not friendly for os manipulation at all
If you say so. I'd love to hear how did you get to that conclusion.
> and number crunching power is not needed in many, many tasks of that sort.
You are aiming very wrongly, it's about startup time. I got sick of Python's 300+ ms startup time. Golang and Rust programs don't have that problem.
I'm not sure where you're going with this: My experience of Ruby and Go is that:
1. Go is a lot easier to do OS manipulation type stuff.
2. Go is a lot easier to modify down the line.
TBH, #2 is not really a consideration for shell-scripts - the majority of the time the shell script is used to kick off and monitor other programs, transforming an exact input into an exact output.
It's glue, basically, and most uses of glue aren't going to require maintenance. If it breaks, it's because the input or the environment changed, and for what shell is used for, the input and the environment change very rarely.
a) put a `binding.irb` (or `binding.pry`) in any rescue block you may have in your script - it'll allow you to jump in and see what went wrong in an interactive way. (You'll need a `require 'irb'` in your script too, ofc)
b) I always use `Pathname` instead of `File` - it's part of the standard library, is a drop in replacement for `File` (and `Dir`) and generally has a much more natural API.
c) Often backticks are all you need, but when you need something a little stronger (e.g. when handling filenames with spaces in them, or potentially hostile user input, etc), Ruby has a plethora of tools in its stdlib to handle any scenario. First step would be `system` which escapes inputs for you (but doesn't return stdout).
d) Threads in Ruby are super easy, but using `Parallel` (not part of the stdlib) can make it even easier! A contrived example: `Parallel.map(url_list) { |url| Benchmark.measure { system('wget', url) }.real }.sum` to download a bunch of files in parallel and get the total time.
MacOS has Ruby 2.6 installed by default which is perfectly serviceable, but it's EOL and there are plenty of features in 3+ that make the jump more than worthwhile.
irb is a part of Ruby core, so this isn’t true. (It may have been at one point? I’m not sure.)
I love binding.irb. I use it all the time.
I've mostly been in the Python ecosystem for the past few years and the LSP investment from Microsoft has really shown. Rich Python support in VSCode is seamless and simple. Coming back to Ruby after that caught me off guard - it feels like I'm writing syntax-highlighted plain text. There's an LSP extension from Shopify, but it's temperamental and I have trouble getting it working.
Editor support isn't everything (the actual language design is still the most important), but it definitely affects how eager I am to use it. I basically never choose Ruby over Python given the option, which is too bad. Ruby's a cool language!
> I was trying to set up editor support
Not sure what problems you had exactly, but saying that editor tooling is bad, simply because you can't get it to work, is not fair. I've been using the LSP from Shopify since it came out, it works great, is very stable and updates come in on a regular basis.
Sorry, but calling it "a mess" simply because you can't get it to work is quite unfair. I've been using the LSP from Shopify since it came out, it works great, is very stable and updates come in on a regular basis.
Then you think “maybe I just have the wrong lsp” only to realize there are half a dozen that all behave differently and nobody can agree on.
I tried them all, they all turned even my simplest of scripts between 10-50% red. I think half of my ire towards python came from the fact that the LSP situation was so awful, I just had to get used to reading code with a bunch of “errors” in my face, or turn them off completely… I could never decide which was worse
I never search _exactly_ why Ruby became so less popular than Python, but I think that at least two things:
- Having its popularity too much dependent of Rails; - Not having any other killer app (e.g. Python is not only Django, but NumPy, TensorFlow, Pandas, Flask and so on);
So, even if I love Ruby, Python is still my main language for most of things as it has a huge ecosystem and it is easy to use. But for writing shell scripts the advantages that I mentioned in the article generally I don't care too much about those things.
What's between the backticks is not even portable; the commands rely on an operating-system-specific command interpreter.
> puts `ls`.lines.map { |name| name.strip.length } # prints the lengths of the filenames
Fantastic example, except for the commandment violation: "thou shalt not parse the output of 'ls'"!
You really want to read the directory and map over the stat function (or equivalent) to get the length.
2> (flow "."
open-directory
get-lines
(mapcar [chain stat .size]))
(727 2466 21 4096 643 16612 5724 163 707 319 352135 140 51 0 4096
114898 1172428 1328258 4096 4096 4096 29953 4096 4096 0 27 4096
4096 35585 8450 968 40960 14610 4096 14 755128 1325258 4096 17283
218 471 104 4096 99707 1255 4096 129 4096 721 9625 401 15658
4096 235 98 1861 664 23944 4286 4096 1024 0)No thanks. Anything that doesn't have error handling enabled by default goes straight in the trash bin.
The closest I could find is what you suggest (checking $?), or using something like this [1], which would require changing syntax:
system('ls -j', exception: true)
Some would say calling out to shell is an anti pattern by itself. Others would say exceptions are an anti pattern. (Just use appropriate return type and there's no need for exceptions ever!)
There’s really no need for this kind of over-the-top response.
This isn’t true for Ruby nor shell scripts. In Ruby you have `system` or `Open3`. In shell scripts you:
if my_command
then
on_success
else
on_failure
fi
I believe the author was talking about set -e (often used with -o pipefail), so that any unhandled error simply exits the script.
I don’t need to check the exit status of every command.
> I believe the author was talking about set -e (often used with -o pipefail), so that any unhandled error simply exits the script.
I have no idea how you could get that impression from the section I quoted.
How would setting one option once at the top of the script mean “Requiring the user to manually check $? after each command”?
1. Make it difficult to ignore that a function can return an error. This is the golang approach. Errors are part of the return values and unused return values are compiler errors.
2. Make it impossible to use parts of the return value having an error state. Rust does this with the Result sum type and pattern matching.
3. Tailor for the happy-path and throw exceptions in case there are any errors. With optional means to catch them if recovering errors is desired. This is how most other languages function.
Hiding the error status in another variable, that is super easy to overlook and that the programmer might not even know exists, then continuing despite this variable not being checked will inevitably introduce bugs allowing faulty data in your system.
Indeed. Equivalent in Ruby:
Dir["*"].map { |x| File.size(x) } (flow (glob "*") (mapcar [chain stat .size]))
None of this makes any sense to me, and I write Ruby for my day job.
Except from Bash (where backticks also have the same purpose as Ruby), I only remember seeing backticks in:
- Kotlin, for naming functions with phrases. The only use case that I remember for it was to creating more meaningful names for test functions. I don't think that it was so useful... - Lisp dialects for quasiquotes, which is meaningless in Ruby - Haskell, for making functions infix, and I can't see why would it be useful in Ruby (Ruby has it own way to make infix methods) - JS, for creating templates. In Ruby we can use double quotes for that
> "thou shalt not parse the output of 'ls'"
Yes, but it was only an example of associating backticks and the language features. Of course it is not ideal, but it is an example that everyone will understand. You'll probably won't want to use it (even because you can do it with Dir.glob('*').map { |f| f.length }).
The same happened later in the text when I used a regex to find the Git branch.
> You really want to read the directory and map over the stat function (or equivalent) to get the length.
But that is not Ruby.
Since that document, Python PEP 723 was approved, see https://peps.python.org/pep-0723/ and https://packaging.python.org/en/latest/specifications/inline...
Similarly, Rust RFC 3502 was approved and an unstable implementation is available, see https://rust-lang.github.io/rfcs/3502-cargo-script.html
#!/usr/bin/env nix-shell
#!nix-shell -i bash -p cowsay imagemagick
identify "$1" | cowsay
Another common pattern I see is people using embedded ruby in a shell script. It does make it a little harder to read/understand at a glance, but it's nice for being able to do most simple things in sh but drop into ruby for things that sh/bash suck at.
That said, I get a feeling that the people that joined once we'd added stuff outside of the Rails monolith and don't know/use Ruby are...not big fans.
Ruby is a wonderful language worth learning (and it's really not difficult to pick up) but I see way more people push against learning it than they do other things (eg python).
Small nit: your note in Feature 4 is actually supposed to be in Feature 5, I assume.
So for me, "is it installed in the base distribution" is the difference between being able to start immediately no matter which box I'm concerned with, and spending months trying to upstream a new program to be installed with our OS image team.
I took a look around a vanilla Debian 12 box, and didn't see Ruby in there [1]. So, sadly, although I really like the way Ruby looks, I'm going to have to stick with Bash and Python 3 for the hard stuff.
[1]: https://hiandrewquinn.github.io/til-site/posts/what-programm...
For me, when Bash ain't enough I upgrade to the Project language (PHP, Python, JS, etc). For compiled project I reach for LSB language (Perl, Python) before introducing a new dependency.
https://refspecs.linuxbase.org/LSB_3.2.0/LSB-Languages/LSB-L...
Do people really write scripts against Python 2 just so that they're guaranteed to be supported by LSB?
Assuming you can survive all the incompatible interpreter changes for Python etc., the main annoyance with LSB proper is shared library versions.
In my professional experience on medium to large-sized projects, its lack of explicit typing, the habit of Rubyists to use its fairly-substantial metaprogramming capabilities at the drop of a hat, and (for projects that include pieces or all of Rails) the system's implicit library loading make such projects a nightmare to reason about and maintain.
I'm sure these aspects are manageable with a small group, or a very, very disciplined group, but when you have large-sized projects that have been continually worked on for ten, fifteen, more years, you're just not going to be able to guarantee that everyone involved will have the combination of knowledge and discipline require to ensure the thing remains easy to understand, extend, and maintain.
What parts of what I said is not true, and how?
I'm not sure what you mean by this? Ruby gives you pretty much the same namespacing powers that most mainstream languages do.
If your argument is that noone should EVER be able to hoist something into the global namespace, then I'm going to have to pretty strenuously disagree with you. Power tools are good to have for when you need them... but it's very important to have the restraint to only use them when you need them and leave them on the shelf when you don't.
Contrast that with gitlab-runner (Go) or VSCode (Typescript) both of which I have been able to not only read easily but contribute features to. VSCode is at least as big a codebase as Gitlab.
That experience has made me want to avoid Ruby like the plague!
It was only after working on other projects in other languages (Clojure, Java, Scala, Nodejs, Elixir, Rust) that I started to realize that maybe not all languages lead to teams writing code that is this difficult to follow.
People always say "oh, you can write terrible code in any language", and while this is true, it's a tautology. It doesn't actually tell you anything useful. I now think there is actually a pretty large spread in what kinds of code the various languages/frameworks encourage people to write. I'm not saying it guarantees what kind of code people will write, but, just for example, there absolutely is a difference between what the average Clojure programmer will write and what the average Java programmer will write.
If all of the various languages and frameworks and libraries all just ended up with the same effort producing the same results, no one would ever make anything new, because it would be pointless to do so.
If the enormous pile of Ruby projects that I and the folks I work with at $DAYJOB had been the things I was required to work on and maintain at my first dayjob, I would have (no joke) left the field to go become a lawyer.
I'm so, so, so glad that I started off with C++ and got to work at a couple of C++ shops which got me to understand in my bones the importance of good tests and comprehensible-to-a-mere-mortal code.
> ...there absolutely is a difference between what the average Clojure programmer will write and what the average Java programmer will write.
Oh yeah, definitely. After being worn down by the many years I've burned at my current position, I've become VERY skeptical of the "Let's use $LANGUAGE_OR_TOOL because it will make it easy to hire!" argument. The quality of the stuff that the average user of that tool produces matters A TON.
(Which is not to say X is flawless even at scale or a clear best fit along all axes. That is true for no X that I know of.)
That hasn’t stopped billions of dollars of revenue from being created with it.
At least Ruby is unpopular enough (compared to Node) that people who know it are probably decent at their job.
[0] https://canonical.com/blog/canonical-releases-ubuntu-24-04-n....
Not on my Fedora box either.
Luckily, I've found that Perl has most of the best features of Ruby, and it's installed everywhere. It's time to Make Perl Great Again.
Please name those 10 environments you are talking about. In my experience, a reasonably recent Ruby version is present almost everywhere.
> add 5 minutes to your docker build
Why on earth would it take 5 minutes to install anything? If you install Ruby through a package manager (it's present in pretty much all of them: https://www.ruby-lang.org/en/documentation/installation/#pac...) it takes only seconds.
You mention using threads and regex match global variables in the same write up. Please use the regex match method response instead of the $1 variables to save yourself the potential awful debugging session. It even lets you access named capture groups in the match response using the already familiar Hash access API. Example: https://stackoverflow.com/a/18825787
In general, just don’t use global variables in Ruby. It’s already SO easy to move “but it has to be global” functionality to static class methods or constants that I’ve encountered exactly zero cases where I have NEEDED global variables. Even if you need a “stateful constant” Ruby had a very convenient Singleton mixin that provides for a quick and easy solution.
Besides, if you actually WERE to take advantage of them being global VARIABLES (reassigning the value) I would confidently bet that your downstream code would break, because I’m guessing said downstream code assumes the global value is constant. Just avoid them, there’s no point, use constants. This applies to any language TBH, but here we’re talking about Ruby :)
Even though I wrote this about Ruby, I must admit that I am not a specialist.
People use it a lot less these days, for a lot of reasons, some better than others. I myself do simple stuff in bash, and pull out some Python for more complex scripting. My Perl chops have withered, last time I was paid to use it was 21 years ago, but it really is a great scripting language, and you'll find it installed on a great deal more systems than Ruby.
One of these days I'll give Raku a spin, just for old time's sake.
For example: Ruby has no built-in for "call a subprocess and convert a nonzero exit status into an exception", ala bash `set -e`. So in many of my Ruby scripts there lives this little helper:
def system!(*args, **kwargs)
r = system(*args, **kwargs)
fail "subprocess failed" unless $?.success?
r
end
class << ENV
def path
@path ||= self['PATH'].split(':').map{ |d| Pathname.new(d) }
end
def which(cmd)
cmd = cmd.to_s
self.path.lazy.map{ |d| d + cmd }.find{ |e| e.file? && e.executable? }
end
end
• implicitly logging process-spawns (ala bash `set -x`)
• High-level wrapper methods like `Pathname#readable_when_elevated?` that run elevated through IO.popen(['sudo', ...]) — the same way you'd use `sudo` in a bash script for least-privilege purposes
• Recursive path helpers, e.g. a `Pathname#collapse_tree` method that recursively deletes empty subdirectories, with the option to consider directories that only contain OS errata files like `.DS_Store` "empty" (in other words, what you'd get back from of a git checkout, if you checked in the directory with a sensible .gitignore in play)
...and so forth. It really does end up adding up, to the point that I feel like what I really want is a Ruby-like language or a Ruby-based standalone DSL processor that's been optimized for sysadmin tasks.
Since Ruby 2.6 you can pass `exception: true` to `system` to make it behave like your `system!`.
https://rubyreferences.github.io/rubychanges/2.6.html#system...
But come to think of it, if Kernel#system is just doing a blocking version of Kernel#spawn → Process#wait, then shouldn't Process#wait also take an exception: kwarg now?
And also-also, sadly IO.popen doesn't take this kwarg. (And IO.popen is what I'm actually using most of the time. The system! function above is greatly simplified from the version of the snippet I actually use these days — which involves a DSL for hierarchical serial task execution that logs steps with nesting, and reflects command output from an isolated PTY.)
Unbelievably easy to read, and, with rspec, it is stupid easy to write tests for. No need to fuss with interfaces like you do with Golang; yes, that is the right thing to do, but when you need to ship _now_, it becomes a pain and generates serious boilerplate quickly.
I've switched to Golang for most things these days, as it is a much safer language overall, but when shell scripts get too hard, Ruby's a great language to turn to.
irb
~ % irb
WARNING: This version of ruby is included in macOS for compatibility with legacy software.
In future versions of macOS the ruby runtime will not be available by
default, and may require you to install an additional package.
irb(main):001:0>Involving an entirely separate parser just to split your args up feels like a footgun to me.
Python's `sh` handles this rather nicely:
Moreover, how often do you really move a script to a completely different OS, where you don't know which OS libs are installed? And wouldn't those missing OS libs also be a problem when writing the script in Bash or any other language?
I need to check it because my mother language is Portuguese and my personal page was initially only in Portuguese (everything there has also a version in Portuguese if you click the Brazilian flag). Some years ago I started to write in English but I didn't want to delete my older posts.
`foo + bar` and `foo+bar` are `foo()+bar`, but `foo +bar` is `foo(+bar)`
ternary ? : also has some interesting whitespace dependent mixups with symbols, but I cannot remember what. I think that parser has many gotchas like that, but they are really really rare to bite you, because ruby's magic follows human intuition as much as possible.
$ irb
irb(main):001:0> def foo(x=70) = x
=> :foo
irb(main):002:0> i = 2
=> 2
irb(main):003:0> foo / 5/i
=> 7
irb(main):004:0> foo /5/i
=> /5/i Pathname.glob('*').filter { |f| f.file? }.each_with_object({}) { |f, h| h[f.size] = f }
result = {}
for file_path in glob.glob('*'):
if os.path.isfile(file_path):
result[os.path.getsize(file_path)] = file_path
string.split(' ').map(&:capitalize).join(' ')
words = string.split(' ')
capitalized_words = [word.capitalize() for word in words]
result = ' '.join(capitalized_words)
' '.join(map(str.capitalize, string.split(' ')))
result = {os.path.getsize(f): f for f in os.listdir() if os.path.isfile(f)}
result = ' '.join(word.capitalize() for word in string.split(' '))
result = ' '.join(map(str.capitalize, string.split(' ')))
result = string.title() from string import capwords
result = capwords(string)
The file example you gave could also be turned into a dict comprehension if desired. I’m on mobile, but I think this would work.
result = { f:os.path.getsize(f) for f in glob.glob(“*”) if os.path.isfile(f) }This is probably easier.
A scripting language needs a way to declare dependencies in a locked-down way, inside of the script that requires them. They must be portable across platforms.
Nice calling syntax though.
For example:
require "open3"
last_stdout, wait_threads = Open3.pipeline_r("cat /etc/passwd", ["grep", "root"])
last_stdout.read # => "root:x:0:0::/root:/bin/bash\n"
wait_threads.map(&:value).map(&:success?) # => [true, true]
Generally marshalling a gig or more of JSON (non-lazily) takes a lot of resources in ruby.
Is lazy marshalling something that other languages handle better?
With file as ndjson it was easier, if a little sparsely documented (Zlib::new or #wrap?):
my_it = Zlib::GzipReader.wrap(some_ndfile).lazy
obs = my_it.each_line.lazy.map do |line|
JSON.parse line
end.first(4)
My issue is more that it is tricky to nest ruby IO objects and return a lazy iterator - especially nesting custom filters along the way - at least more tricky than it should be.
Apparently there's a third party frame work that does seem promising:
https://iostreams.rocketjob.io/tutorial
Or manual lifting:
https://dev.to/bajena/streaming-gzipped-csv-files-from-ftp-i...
Or:
https://medium.com/smartly-io/streaming-data-with-ruby-enume...
https://github.com/lautis/piperator
I think something more like this should probably be built in, and readily available (for gzip, http, files etc). Maybe I'm greedy.
Btw the shell pipeline to convert a file would be something like this, and is fully streaming:
# gzipped JSON to gzipped ndjson, stripping top level array:
gzcat file.json.gz \
| jq -cn --stream 'fromstream(inputs|(.[0] |= .[1:]) | select(. != [[]]) )' \
| gzip -9 \
> file.ndjson.gzip last_stdout, wait_threads = Open3.pipeline_r(
["gzcat", "some.txt"],
["grep", "foo"],
["sort", "-u"],
["head", "-10"],
)
[0] https://github.com/ruby/open3/blob/b8909222051b4103a19eba195...
loosely pinned deps installed on execution sounds fucking awful.
`gem "mygem"` installs the latest version. `gem "mygem", "~> 4.0.0"` installs >= 4.0.0 but < 4.1.0, which is what you probably want when using Semantic Versioning, which most gems adhere to, to get the latest patch version. `gem "mygem", "4.0.10"` installs exactly that version.
I haven't used backticks in my shell scripts in years.
Yeah, I understand... But this is other language with a similar syntax for a similar functionality.
One can mitigate the problem somewhat using the `--disable-gems` flag, but that's not a good general solution.
[borg@cube] time ruby -e 'nil'
real 0m0.007s
$ ruby -v
ruby 3.3.1 (2024-04-23 revision c56cd86388) [x86_64-linux]
$ time ruby -e ''
real 0m0.122s
user 0m0.102s
sys 0m0.020s
Of course, if you need more peformance, you need to go for newer stuff.
Actyally that miniruby is pretty usefull. For basic stuff. Also, its very easy to build static ruby containing all extra stuff you care about, like Win32 API for example.
I myself have custom ruby binary on Win32 (Cygwin) with GRX library added in, So I can do basic graph stuff directly from ruby :) That stuff is written in it:
> For basic stuff. Also, its very easy to build static ruby containing all extra stuff you care about, like Win32 API for example.
Wow, this reminds me abit of MRuby. So I could basically ship a script with the self-contained ruby executable instead of having to force users to install Ruby on their machine? How can I get ahold of miniruby? Or is there any resource somewhere that I can dig into?
Hard to say about new Ruby versions like 3.x or even 2.x. 1.8.x have pretty simple building process, just grab the old source. You can use --enable-static builds there. Miniruby is build by default, always, because its part of building process.
Also, if you are platform are you targeting? Win32 only?
I wonder if experimenting miniruby with cosmopolitian would help with that?
But Idk, I’m just thinking loudly and probably missing a huge detail which would make the idea non-working.
Also, thanks for the linked resource, I’ll definitely give it a go!
> That is, most of the cases Bash for me is enough, but if the script starts to become complex, I switch to Ruby.
Even if ChatGPT lets you bang out more complex shell scripts easily, if you have to come back to it later on to fix an error or add a new feature, it's really hard to understand it (if you don't deal with such scripts on a daily basis). If you start with Ruby (or Python or similar) from the beginning, it's much easier to understand and extend later on.
Ruby is quite similar to Python, perhaps you don't even need to study too much!
As a Python guy I found the setup for this sort of CLI too really refreshing!
Being able to just paste json into a terminal and immediately start working with it in a structured way feels like a superpower.
The interactive help and well-thought-out error messages feel like a hug.
Too many ways to do the same thing.
Not packaged by default on most Linux.
Monkey patching makes things even harder to debug.
whereas the origin of monkey patching seems to by Python with its totally broken magical method names and kitbashed object model.
> Too many ways to do the same thing.
In Bash you can use backticks, $(), and bash -c for the same thing. And also [ ], [[ ]], test. And so on.
> Not packaged by default on most Linux.
You can install it. Some distros (e.g. Alpine) even don't ship Bash. If you are restricted to only what comes by default in an operating system, then probably a simple script written in sh may be the solution.
> Monkey patching makes things even harder to debug.
Just because you can doesn't mean that you should. I won't label something as "hard to debug" for features that I don't need to use, I label as "hard to debug" for features that I _need_ to use. Then I can say: Bash is hard to debug.
Python > Lua > PHP > Ruby > JavaScript > Perl > Bash
Those are different languages, with different characteristics. Perhaps the only two that can be directly compared as a 100% replacement to another are Ruby and Python.
If you enjoy it, more power to you. However, Python is everyone's second favorite or least favorite language, and it runs laps around Ruby any day. Then there's Go if you need some extra oomph!
In my professional experience, Go cannot consistently avoid producing binaries that fail when run on an incompatibly-different version of glibc than they were build and linked on.
Also. for most deployment situations, Go's "single, enormous binary" doesn't matter.
* In The Cloud(TM), you have full control over what you deploy. So, so, so often you have a custom VM or Docker image that you just squirt out there and automation to keep it updated and rebuilt, which makes deployment just trivial.
* On Windows, you have Windows Installer, which you can instruct to check for and install any prereqs you require. (It's incredible how all of the pieces to build a proper package manager have been in Windows since the early 2000s, and yet no package manager came out of MSFT.)
* On OSX, you either use the App Store or one of the handful of package managers... but I expect nearly zero non-Power-Users use the package managers. I guess there are also the nutballs who do 'curl $INSTALL_SCRIPT | sudo bash'.
* On Linux... well nearly zero non-Power-Users run desktop Linux, and those that do likely already have Ruby, Python, Perl, and GCC installed.
I think you are commenting on Rails, not Ruby when you refer to "convention over configuration". Even in Rails I'm not sure what "esoteric" means in your comment and that approach isn't even related to the object-oriented concepts.
Is today’s Python much faster than, say, 5 years ago? Ruby was definitely quicker than Python back then.
I’ve heard of performance improvements to Python, but they seem to be 10-25% range which wouldn’t be enough to catch up to Ruby.
Also I think the convention over configuration mantra belongs in the Ruby on Rails world rather than Ruby itself. Ruby as a language is far less opinionated about how you do things, especially for small shell scripts.
1: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
We're talking about code that calls external commands here. If one wants performance it is already doing wrong by calling external commands. Don't think it is relevant here.
> encourages an esoteric convention over configuration style of OO code.
I can't see that. Its OO is not so different from other languages. Perhaps you are thinking about Rails (as I said in the first paragraph).
Language is just a tool, anything and everything works if you love it enough