I got tired of PHP and Perl, so I tried bash
github.com
github.com
I think the "big realization" for me was shifting to think in pipelines instead of passing state around in variables. That completely removed the pain of no arrays in POSIX shell for me. Stdin/stdout are your friend.
Certainly, shell scripting isn't the answer for everything, but with shellcheck (and bats!), I feel like it's a really reasonable default for systemy things. Heck, the status bar I use in my window manager is essentially just a 700 line bash script, with asyncronous update and everything. It's even quite readable and orthogonally organized, if I might boast. Have been pleasantly surprised over the years every time I go back and read it.
https://www.in-ulm.de/~mascheck/various/set-e/
And those are just POSIX compliant shells. I mean this doesn't matter if you just want to write a script for yourself but becomes frustrating when you want to level up you shell skills and write cross-platform scripts ;-)
That said, I have to add that I love shell scripting... I am not exactly sure why, but I guess it is because it lets you build powerful things with very few lines of code.
With JS that is different. Yes, every browser has a few extra bits here and there, but there is very little that many browsers have that is not part of the standard. With compilers, it is different because the developer can decide which one he wants to use at build-time. After that, his program still might behave differently on different platforms (depending on APIs, etc.) but in general, the language that he uses behaves the same. With shell scripts that is not the case as the behavior depends on the run-time shell and the developer has very little influence on which shell will be used.
So while you are right that every programming language has this problem, some are better at managing it than others and shells are particularly bad at it in my opinion.
Oh gosh, it was such a pain work around those old bugs...
Does someone know why Apple doesn't update the bash?
Now there was one offshoot that tried to get better at longer scripts: The Korn Shell. In it's later edition (ksh93) it actually had hashes and a pretty good way of expanding its functionality. Heck, dtksh would've given Tcl/Tk a run for its money in the 90s if both ksh and Motif (the library dtksh used for its widgets) weren't proprietary at the time.
I worked on a "devops" system in the early '00s that was used to provision all kinds of Unix servers and workstations (including HP-UX, AIX, Solaris, Linux, BSD..), and the major parts of it was written in ksh93. It worked quite well.
I still think that all of this is inferior to both Perl5 and Tcl/Tk, but ksh93 made some decent strides way back when.
Oh yeah, and Motif can die in a fire.
"There are many ways to skin a cat" is older than computers, and it applies to basically any programming language. Some languages make it harder to do it in a different manner, some allow it, but don't offer it without including some additional package. C has no hash tables, but it's not that hard to get them, for example.
I'd go so far as to say, that plenty of current programming languages allow more ways to do a thing out of the box than Perl. What mythical language does offer a straight and completely obvious path from problem to solution? Haven't I ascended enough in the halls of programmer-dom to be told of this coding panacea?
And why would this be particularly bad for DevOps? Are DevOps programmers worse and/or more easily confused?
Just recently we had a thread here about different ways to remove duplicates from arrays, and that was done in Golang, devops Holy Grail du jour.
DevOps is all about consistency and readability... neither of which are strong points of Perl. Perl was the de-facto cross-platform DevOps language in the 90s (seriously quite a few Mac programs of the day installed Perl as a dependency). There’s a reason the industry moved away from it 15 years ago.
I’ve been building in Perl5 for over 20 years, and it’s really easy to paint yourself into an edge case where things don’t quite work the way you expect them to. Everyone who writes Perl learned it a slightly different way, which makes managing an active codebase of Perl “fun” — it has a tendency to devolve into spaghetti code over time once you get a lot of contributors.
Again, what weird single-approach programming language are we comparing Perl5 with?
I'd fear though that having BATS would just increase the tendency to let a bash script grow from 50 lines through to 200, when it should've just been ported to Python at 100.
For pipe-and-filter system management use case, I tend to think that Powershell is the peak of the shell evolution currently, but that is probably my limited insight. When I learnt shell scripting, zsh was the best available :) but I preferred the ksh.
That said, there are niches where bad error handling isn't as much problematic. Ironically, Bash is much more fit for web programming than for system scripting.
ever since I started dabbling in racket I've grown fond of not dealing with state and look at evreything like a function. (I'm not a functional programmer).
The old UNIX philosophy with pipes is very close to that as in every command in the pipeline is a function in similar way. I'm not saying hacking together a bash script is FP but it's still possible to apply the FP thinking in the UNIX pipeline better than building a monolith that handles the whole pipe. I can pass state around (env vars) but it's an extra effort and it forces me to think whether I can get it done without state first (via optimizing the input output formats).
Could you elaborate please? I see that data - especially sequential, array-like data - is conveniently passed via pipes (and e.g. jq https://stedolan.github.io/jq/ uses this concept), but I feel you're saying something deeper or a different kind of use-case... [perhaps because you say "state" not "data")
A big part of this was figuring out I could the `read` builtin to parse stuff into local variables:
if read -r foo bar baz _; do
# do stuff with $foo, $bar, $baz
# $_ contains anything extra on the line
done
echo $some_local variable | unix_tool
Anyway, using `read` works especially nicely with tabular data too. For example, a naive CSV parser might look like
while IFS=',' read -r field1 field2 field3 _; do
# do stuff
done while IFS=',' read -r field1 field2 field3 _; do
# do stuff
done
Fish is better but still looks legacy.
Encouraging this is why bash scripts scare me.
Because "return values" in bash are implemented by echoing (printing to stdout):
return=$(my_function arg1)
result=$(my_function arg1)
result_a=$(echo "$result" | grep A)
result_b=$(echo "$result" | grep B)All of the later scripting languages aren't really meant for writing shell script as invoking commands is just tedious instead of just using back ticks in perl.
Talk about going in circles of history.
Every time to reach for an array in bash, try a pipe instead.
Every time you desire multidimensional arrays, pipe through filters.
(At least that's what I think you're saying.)
I am glad people prefer Python or Ruby for more complicated stuff and Go, Java or compiled langages like Rust, C/C++ for performance-intensive jobs.
Perl is fast but the syntax is terribly non-intuitive and I would mark it 'do not use for new designs'.
bash.js has a nice ring to it
WebAssembly + Ring-0 = Security /s
It's a huge bash script for parallel processing of work. It was great fun and I'm proud of it as I don't see myself as a programmer of any sort.
But gosh since I learned Powershell and Python, I would never ever ever ever ever use Bash for anything substantial.
If you ever feel like you grow 'fond' of Bash or Shell scripting, it's time to seek a doctor. Please learn Python or some equivalent instead.
-----------------
Edit: reading the code of this web server project:
Let's be absolutely clear: this is a joke project. DO NOT USE THIS FOR # ANYTHING, ESPECIALLY ON A PUBLIC FACING SERVER.
from pyshell import sh, ShellError
def func(a,b)
output, retcode = sh("df -k | awk '{print $5}'")
if retcode == 0:
return output
else:
raise ShellError(output,retcode)I was also planning to extend it to support python3 asyncio, but never got around it
It provides an idiomatic shell scripting experience for python.
For example, it uses the "pipe" operator overload to allow complex pipelines to be created (pipes are created using platform native pipes, so there's no performance difference from a shell pipeline). The same operator can also be used for redirections
By the way, why not put the raise inside sh?
EDIT: Ok, the difference is that os.system does not return stdout. But you can use the subprocess package to capture stdout (and stderr as well). If you want pipe syntax, you can invoke e.g. bash -c "...", where the dots are your command. But the problem with this is that it's prone to injection attacks, unless you are very careful. I guess the same warning applies in your original "sh" example.
>>> from invoke import run
>>> date = run("date")
Sat Apr 13 11:09:18 PDT 2019
>>> dir(date)
['__bool__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__nonzero__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'command', 'encoding', 'env', 'exited', 'failed', 'hide', 'ok', 'pty', 'return_code', 'shell', 'stderr', 'stdout']
>>> date.return_code
0
>>> date.stdout
'Sat Apr 13 11:09:18 PDT 2019\n'
>>> do_they_exist = run("grep banana /etc/passwd")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/pgraydon/Envs/python3/lib/python3.7/site-packages/invoke/__init__.py", line 46, in run
return Context().run(command, **kwargs)
File "/Users/pgraydon/Envs/python3/lib/python3.7/site-packages/invoke/context.py", line 94, in run
return self._run(runner, command, **kwargs)
File "/Users/pgraydon/Envs/python3/lib/python3.7/site-packages/invoke/context.py", line 101, in _run
return runner.run(command, **kwargs)
File "/Users/pgraydon/Envs/python3/lib/python3.7/site-packages/invoke/runners.py", line 271, in run
return self._run_body(command, **kwargs)
File "/Users/pgraydon/Envs/python3/lib/python3.7/site-packages/invoke/runners.py", line 404, in _run_body
raise UnexpectedExit(result)
invoke.exceptions.UnexpectedExit: Encountered a bad command exit code!
Command: 'grep banana /etc/passwd'
Exit code: 1
Stdout: already printed
Stderr: already printed
http://ibmsystemsmag.com/aix/tipstechniques/systemsmanagemen...
https://redpanthers.co/different-ways-to-run-shell-commands-...
Perl/awk like one-liner/filters in ruby:
http://reference.jumpingmonkey.org/programming_languages/rub...
https://nithinbekal.com/posts/ruby-sed-awk/
Of note is that ruby allows "backtick execution":
filename = 'some_file.txt' wordcount = `wc --words #{filename}`
I find that much of the strength of bash/shell comes from the surrounding utilities though. So simply jumping to ruby won't magically let your "shell script" work on both windows and Linux, just because you have ruby installed;there likely not be a similar set of building blocks like "wc" etc. You could of course replicate most via the ruby std lib - and it's arguably easier to write test driven ruby than bash.
qx(some command some parameters some variables)seems a lot of effort and iterations went into this?
I have seen quite a bit of legacy plumbing like this in production / automation until better (or at least standardized) tooling came along with the agile/devops movement. A lot of this plumbing survived surprisingly long (which is also scary). Often code like this matures over long periods into stable solutions that could even deal with errors well. Makes them hard to replace.
I agree it doesn't belong in production but it looks pretty neat for small internal stuff even in 2019.
Learning from books doesn't work, you need to have a project with a goal to apply it in.
The horrible honest truth is that the world actually doesn't run on bash, python, C or Java.
It runs on Excel macro's.
Your project is a great read!
I must apologize to Perl. I see now where it was coming from [1], and what it saved us from.
May god have pity on your soul for inflicting us with this.
[1] There are lots of quirks in Perl that I now recognize come from shell scripting. Most obviously, $VAR notation and string quoting.
Edit: now that I read the readme I think the title should have been "show hn: I made a web framework in bash" this is pretty neat, I still wouldnt use it, but its cool as hell.
Hmm. I have a slightly different view. If logic is complex, then it should be moved to a different language. But if there are a lot of flat if statements then shell is fine.
$ cat meh.bash
#!/bin/bash
set -e
die() {
echo "Failed at line $1: $2"
}
trap 'die $LINENO "$BASH_COMMAND"' ERR
echo a
test 1 = 3
echo b
$ ./meh.bash
a
Failed at line 8: test 1 = 3
By the way, if you are just using Bash (instead of POSIX /bin/sh) you can do even better. Some time ago I found a stacktrace function somewhere:
set -Eeuo pipefail
trap stacktrace EXIT
function stacktrace {
if [ $? != 0 ]; then
echo -e "\nThe command '$BASH_COMMAND' triggerd a stacktrace:"
for i in $(seq 1 $((${#FUNCNAME[@]} - 2))); do j=$(($i+1)); echo -e "\t${BASH_SOURCE[$i]}: ${FUNCNAME[$i]}() called in ${BASH_SOURCE[$j]}:${BASH_LINENO[$i]}"; done
fi
}Some additional related projects not yet mentioned:
balls (bash): https://github.com/jneen/balls
czhttpd (zsh; full disclosure, author here): https://github.com/jsks/czhttpd
sh-httpd (ksh): https://www.dartmouth.edu/~rc/classes/ksh/sh-httpd.html
zshttpd (zsh): https://github.com/hkoba/zshttpd
ZWS (zsh): http://www.chodorowski.com/projects/zws/
With zsh you can avoid the dependency of netcat/socat and use the builtin TCP module which is both super cool and terrifying.
Based on ctypes: https://github.com/taviso/ctypes.sh/
(Quote: seebs on slashdot)
No disrespect to ShellCheck, though. It's a fantastic tool.
I'm constantly surprised it hasn't exploded yet.
Could you clarify?
It is very well suited for imperative scripts, but the language breaks down pretty quickly once complexity and the level of abstraction go up; you start relying on obscure syntax, behaviour, and piling up more and more patterns that require full week-long immersion to grasp. Perl is harmless in comparison.
tl;dr if you’re not sure how to roll your own very simple web request/response framework, this is a great intro.
Windows converts and Python programmers are the ones who turned the tide. I suspect that when most read older source code from the Unix universe they don't realize it's almost entirely tab indented because 90% of the source code will still look clean no matter the tab stop (which is of course the very point of hard tabs =)
[1] I never understood why Perl's heredoc construct didn't also adopt this behavior. I always cringe when people use heredoc (even in shell) without indenting the entire body properly. I mean, that's kinda the point of the heredoc--so the data becomes part of the code.
How many requests per second could it serve? What about HTTP/2 support?
I love it.
Initial commit of a really stupid idea.
computerx138 committed on Mar 6I don't recall hearing any serious criticism claiming that it was hard to get things done with (unless you count security).
Manageable is subjective. There's no evidence that it's "just right" or even "timely". Major API cleanup would be appropriate for php 8 but there's still too many people clinging to the idea that PHP would be damaged because someone can't look up how to solve PHP problems with specific versions (ie are stuck in the late 90s).
There's so much that could be done in one fell swoop, the steering council of neckbeards (PHP neckbeards are as bad as it sounds) are responsible for the stunted development of the language. To suggest they are the progressive face of PHP is, at best, revisionist.
There's plenty of features in the docs that say at the top "deprecated in x, removed in y, see also alternative z", so I'm unsure of what problem you're referring to.
We ended up rewriting the entire product in Python.
I've done PHP for more than 10 years and got no problem using it if I have to.
OTOH, I don't want to touch stuff like rails because I don't like it being in the way but love Ruby.
I'd prefer TypeScript these days though.
For example, I guess you didn't know there are PHP and Perl REPLs? And robust OO code is not only possible in Perl, but there's tens of thousands of very reusable Perl modules that have stood up over time much better than Python or Node modules have.
Awful language. Waste of time.
How do you do this in a scripting language without complication?
$ ssh remote.server 'mysqldump db | gzip -c' | gunzip -c | mysql db
It's the power of individual commands though I admit bash sucks as a language but it got invented forever ago, so can't blame it. But we need something better than bash. Fish is close but not too good.