Shellcheck: a static analysis tool for shell scripts
shellcheck.net
shellcheck.net
What a great sanity check for the days when I'm writing something on my own without a second pair of eyes to proof-read it.
BTW, it can be a little hard to figure this out, but if Shellcheck gives you a warning that you want to ignore (because you intended to trigger that behavior), you can put the following comment above the offending line:
# shellcheck disable=SC1234
On paper it sounds like it's equivalent to `on error goto 0`, making a script fail-fast -- which would have been awesome.
Instead, it makes a script fail sometimes for things that are sometimes errors. The rules for how and when are unexpected and unintuitive, several weird cases are described on http://mywiki.wooledge.org/BashFAQ/105
If enabling it just granted a free 50% chance of stopping on any given error, it would have been worth it, but it triggers on false positives as well.
Much like any language, you need to read and understand it to be able to truly write, I think. bash is deceptive in this regard IMO, due to how low its barrier to entry is.
Now if you want to see some truly horrible code, implement a shell script that runs in bash and zsh and does exception printing in both.
This, together with poor error handling is a recipe for disaster: Problems with permissions, insufficient disk space, etc. Instead of stopping when encountering an error, most shell scripts will happily continue break at some other point in time (or worse - destroy valuable data).
Again, advertisement for my side project https://github.com/elves/elvish, a Unix shell with true data structures. Still a WIP though.
Don't get me wrong, I realise it has its flaws and warts, but for me, and comparing to cmd or bash I still think it's very, very much an improvement.
Off the top of my head actual mistakes (the sort that tends to bite many people) include handling of [ and ] in -Path arguments (necessitating -LiteralPath arguments in later versions), and the constant wondering whether something returns a scalar or an array (and an array of one element being unwrapped into a scalar automatically). During my time working on Pash I also noted a few weirdnesses on source code side, most recently and notably LanguagePrimitives.Convert which has a dependency on the currently-executing runspace (which is stored in a thread-local field).
Just FYI. Microsoft tried to address that problem in Windows a long time ago when it introduced Powershell.
Only when it allows the same expressive power over the OS as Lisp Machines, Interlisp-D, Cedar, Oberon have over the running environment.
The only mainstream modern shell that approaches that is Powershell.
Edit: forgot to say good luck for your project
REPL is just a Turing-complete real time interpreter. Which means even the VBA "Immediate" panel in (as seen in MS Office) is REPL. And it means Bash is REPL too.
The question you're raising is whether all REPLs are equal. Lisp machines definitely had more control over the host than VBA does. But that doesn't mean that VBA's immediate panel isn't REPL just because a more powerful example exists.
As for Bash, that's a bit of a weird one because Bash wouldn't be much without the accompanying GNU / POSIX userland. But if you're willing to include a UNIX / Linux userland into scope then Bash has just as much control over the host as Lisp did on Lisp machines. But even without the aid of forking additional executables, Bash can still modify the state of the kernel directly. eg
echo 0 > /proc/sys/vm/swappiness
echo 3 > /proc/sys/vm/drop_caches
As for /proc/sys, not all UNIXes have such features.
In Oberon I could pipe selected text from any application into any command that had a GUI aware type signature, for example.
There are lots of command line hooks for GUIs. Want to copy data to the clipboard from the command line? xclip. Want to pop up a notification in your desktop environment's notification bar? notify-send "hello world!" etc
> As for /proc/sys, not all UNIXes have such features.
That example of mine was clearly taken from Linux - so it goes without saying that most UNIXes would behave different in that specific regard. Even so, they'd still have command line tools for doing the same thing (and to be fair, Linux does too, even with a vaguely-Plan 9 virtual file system)
> In Oberon I could pipe selected text from any application into any command that had a GUI aware type signature, for example.
Well like I said, I'm not trying to say that all REPL's are equal, but most of what you're describing is still possible in at least Linux. I'm not saying it's as intuitive nor "pretty" as it would have been on the Oberon, but it's certainly possible.
To be quite honest, most of what you've been posting on this topic has really just been elitism. And I do actually sympathise with your point as working in Bash can be a complete hateful mess at times (even without comparing it to the old Lisp machines). But that doesn't change the fact that Bash is a REPL environment.
Advertisement aside, there is some inherit unsafety in shell scripts that cannot be easily resolved, namely the unsafety involved in interacting with external commands.
Compared to other scripting languages, the greatest advantage of shell languages is the convenience of interacting with external programs. However, at least in Unix, there are few static constraints you can apply to them. Everything we know is that the program will (probably) parse something in argv which are just bytes, (probably) take something from stdin which are just bytes, and (probably) put something to stdout which are again just bytes; there is no universal method to check that the commands arguments are well-formed, or the input format is correct, or the output format conforms to a certain schema without running the actual program. A solution is to define some kind of static protocols for external programs so that their invocations can be statically checked, but it's already too late.
Why is set necessary? Once you have declaration with var, can't mutation be done without set?
What's your thinking behind making var mandatory for declarations? Safety is obvious, but it seems like terseness is a really big goal for shell programming, especially interactive use.
Also, documentation wise, I don't see how/if you do variable expansion in strings. Same as sh?
var $x = "foo"; if $true { set $x = "bar" }; echo $x # outputs "bar"
var $x = "foo"; if $true { var $x = "bar" }; echo $x # outputs "foo"
There are no variable expansions, but strings are concatenated implicitly when they run together. In sh:
echo "hello $name, welcome!"
echo "hello "$name", welcome!"
You can also remove the need for var (at the expense of no safety for typos) by prohibiting shadowing, like coffeescript does.
I was not aware of the CoffeeScript approach towards shadowing before. I will look into it, but it seems to be a very controversial design choice of CoffeeScript.
Re set: it would slightly complicate your grammar, but I don't think detecting "word space* = ..." would create any ambiguities.
Partially this issue is just about how much you value explicitness/regularity vs. concision.
re "word space* = ...": should this echo an equal sign and $ip, or assign $ip to $echo?
echo =$ipIn my case, I'm treating "=" as not able to be included in an unquoted string literal, and I'm requiring variables to start with $.
So in my shell, this would be a syntax error.
echo =$ip
$echo = $ip
Also consider the following snippet:
: ${SSH:=ssh}
$SSH $host1 'command1'
$SSH $host2 'command2'
That is for the traditional shell part, which is still true in elvish (although due to stricter word splitting semantics use case 2 is different in elvish). Also since in elvish closures are first-class values, it's very intuitive to just call them directly:
var $f = {|$x $who| echo "Hello, "$who"!" }
$f = world # outputs "Hello, world!"
var $f = {|$x $who| echo "Hello, "$who"!" }
call $f = world # outputs "Hello, world!"
$f = world # assigns "world" to $f
A relevant note: Coming up with syntax for a shell language is actually very difficult due to the existence of bare words which greatly limit your inventory of potential operators. For instance one would very likely expect "echo user@example.com" to just echo "user@example.com", so if you give special semantics to "@" or "." it causes confusion and inconvenience. The only safe place you can introduce new semantics is the command, and this forces the language to have a prefix structure. If this reminds you of Lisp, you are correct - due to lack of infix operators lisp has very few restrictions on variable names; but the situation in shell languages is the opposite: due to the liberal use of bare words it is very difficult to introduce new infix operators to a shell language.
My one caveat you can have a little more freedom if you're willing to give up some of the patterns of existing shells, at the cost of unfamiliarity. Of course, you also need to be as expressive as shell so far as possible.
Anyway, "set" isn't a high cost to bear. I don't like it, but that's personal preference, and it clearly helps with the syntax of what you're doing.
var a = 1;
function four() {
if (true) {
var a = 4;
}
alert(a); // alerts '4', not the global value of '1'
}
Still, it's nice that 'var' exists in JS at all. The idea that variable declarations are unnecessary noise and should be elided -- an idea that dates back at least to BASIC -- is, in my opinion, one of the worst seductive ideas in programming language design. Unless your language has only a single global scope (like BASIC), it always causes problems -- and we know that block scope is important for nontrivial programs.
Elvish sounds interesting; I will have to check it out.
I know ls returns a list of files, so I should be able to use that. I don't know foo, so it's basically string -> string or whatever, but if an entrepreneuring spirit does know about foo, he could write an abstraction layer for it.
The trick is making a simple interface for that
Edit: It's really impossible to avoid edge cases. Take find: you can't parse it, because it's just a list of filenames separated by \n. But filenames can contain just about any character. How do you handle /home/bar\n/tmp?
Maybe you just ignore pathological input, but now you're regressing towards the state of bash.
The problem with `find` happens to have a solution (-print0). However it is a PITA in deal with \0-separated strings in traditional shells, unless you pipe it to another command that happens to recognize \0-separated strings.
With elvish you can parse the \0-separated strings outputted by `find ... -print0` into a genuine list - not lines (which are \n-separated strings) or \0-separated strings, but real lists that support indexing, iteration, etc. and there is absolutely no chance that two consequent items will run together or one item will be treated as two. Imagine how fantastic it is to deal with that :)
For instance, `ls` outputs a bunch of lines where each line is supposedly a single file name, but this breaks when some file name contains a `\n`. It is possible to use `ls -b` to escape special characters, but now you have to un-escape the filenames when you pass them to other commands. With elvish it is possible to write a wrapper around `ls` that actually outputs a list (yes there are lists in elvish) of strings and each member of the list can be passed around without un-escaping.
https://github.com/Gabriel439/Haskell-Turtle-Library/commit/...