Imagine a world where Bash supports JSON. | Hacker News Reader

Imagine a world where Bash supports JSON.

qaa.ath.cx

102 pointse1ven49 comments

Wow. I was wondering how the hell TickTick.sh "extended" the bash interpreter grammar, because there is no obvious way to do that. The key is on lines 171-189:

    awk -F '``' '{\
    
      # ... jibbery awk-ese ...
    
    }' `caller 1 | cut -d ' ' -f 3` | sed "s/^/S/g;s/$/E/g" | __tick_fun_parse | bash

    exit

So what this is doing, by Jove, is using the "caller" bash builtin + "cut" to print out the filename of the currently executing script, pass that as an argument to awk, and use awk + sed + the "__tick_fun_parse" function to re-parse the script into something that can actually be executed by bash. The "exit" command then exits the current run-through of the script, because otherwise it will reach all the stuff that would normally cause a syntax error.

This is legit hax.

Jsonpipe is excellent for this. It makes json greppable by putting every piece if data on a new line with its key.

  $ echo '{"a": 1, "b": 2}' | jsonpipe 
  / {} 
  /a 1 
  /b 2 
  $ echo '["foo", "bar", "baz"]' | jsonpipe 
  / [] 
  /0 "foo" 
  /1 "bar" 
  /2 "baz"

https://github.com/dvxhouse/jsonpipe

A similar tool: http://kmkeen.com/jshon/

I'd like to see a language that's as easy to use on the shell as bash (optional quoting, chaining, etc), but is as powerful as the more general-purpose scripting languages (Ruby/Python/TCL/Perl/etc). If no such thing exists I think I'm gonna try designing one.

Check out Object Shell: http://geophile.com/osh.

It takes the idea of commands and pipes, but is oriented around python objects. Usable from the command line, and also has a python API.

It includes integration with the OS (e.g. commands to get files and processes as python objects), databases, remote nodes, and combinations of these, (E.g. submit a query to all nodes in a cluster, pull back the results and integrate them.)

I developed and used this tool while developing a sharded system running on a cluster of nodes with postgres running on each.

Object Shell is GPL. See the page above for examples, documentation and downloads.

There's always Scheme Shell: http://www.scsh.net/

Non-interactive and unmaintained, unfortunately.

Wasn't that sort of the original design spec for perl?

Agreed. Besides Perl, you can also use Ruby and PHP just as easily. At least that works for me. The main selling point is the ability to run shell commands with `command here`. I exclude Python from my list because you have to use the subprocess module, which makes it more complicated.

You could use ipython, which is a python-based shell that doesn't need explicitly calling the subprocess module to launch commands (and a lot more goodies).

I am working on a design for one as well. It's rather tricky, because there are a lot of things you have to balance in order to get a well-designed programming language, especially when you want it to be usable for the sort of things shell scripting is good at, because shell as it exists is basically the opposite of a well-designed programming language and is only popular because it got here first.

It doesn't exist, and yes, it would be very useful.

You are traveling through another dimension, a dimension not only of sight and sound but of mind. A journey into a wondrous land of imagination.

Prepare to enter... the scary door.

Indeed :) In which case, you'd love this: http://www.debian-administration.org/article/A_web_server_in...

Be sure to allow prefixes of ../.. for interesting files.

Or an IRC bot: https://github.com/rpjohnst/markbot :)

Number one thing I learned from this -- don't write code in bash ...

Oh, Man, I can't agree. I LOVE using bash for quick scripting, because I can use the same commands I use when I run them in the shell.

Doing a quick

  for i in `seq 1 15`; do ssh www$i.website.com "reboot"; done

is VERY convenient. It's fast, quick to write, and extends existing shell knowledge.

This would be very useful in all sorts of sysadmin places.

Imaging that I wrote a quick server, in Python/Rails/Node/whathaveyou, that accepted input, then returned a formatted JSON array with server info.

Now, I can do

  $SERVERLIST = `curl http://my-server/list-of-servers`
  for i in ```$SERVERLIST.webservers.list()```; do ssh $i "reboot"; done
  for i in ```$SERVERLIST.dbservers.list()```; do ssh $i "uptime" >> serverload.txt; done

Doing a quick [...] is VERY convenient.

I agree! If you do this a lot then you might be interested in GNU parallel. It's super handy!

    seq 1 15 | parallel ssh www{}.website.com reboot

http://www.gnu.org/s/parallel/

I've found that Perl works even better for this.

my @allfiles = `ls /somedir/`; for my $file (@allfiles) { chomp $file; ... }

Is a wonderful thing. It's not pretty Perl, but it's still better than Bash, IMO.

Dude.

    my @allfiles = glob '/somedir/*';

Now you properly report errors and you don't have to chomp.

I think his point is that he can do the above directly from the command line.

I write tons of little mostly throwaway scripts in bash/sed/awk/etc. But I never had a bash script that grew to reasonable complexity that I wish I hadn't just started in Python.

Each tool has a purpose -- I don't think mucking around with json is particularly suitable for bash...

Have a look at fabric - it makes things like that pretty simple. Also agreeing bash is not a language for serious coding.

You (and everyone else who feels this way) need to take a look at the startup scripts for your favorite Linux distro.

If you mean the repetitive spaghetti of pseudo text processing in init.d, then yes I've seen it. Everything tries to do the same stuff in a different way, there's lack of one framework, error checking is present only in random places and any script can hang because there's no real timeout handling. They're also different in almost every distribution and some are incompatible between sh/bash and very few people can tell the difference. The whole idea is one level too low and we're finally growing up to use something simpler.

That's one of my "favourite" examples of using scripting where it's not needed. Have a look at what's actually keeping critical systems working -> inittab, supervisord, etc. Lately upstart also migrated from scripts to something more declarative.

Edit: forgot the bit where everyone "parses" the output of commands like `ifconfig` and pretends they will never change, or will never fail, or will return the information in the same format. How many scripts relied on grepping ifconfig output for "inet" before "inet6" broke the startup?

Ultimately the point was that shell scripting shouldn't be used for "serious" programming. Yet it has been used for decades to accomplish and automate some of the most important tasks on UNIX/Linux systems. I happen to know and work in several "serious" programming languages, and yet I can knock out a functional, reliable, and understandable shell script in a fraction of the time it would take many people to do the same in X.

Since it's tracking on HN, I'll also point this out: http://www.leancrew.com/all-this/2011/12/more-shell-less-egg...

You mean the scripts that start thousands of processes before my computer is even ready to use? Especially things like sed, awk and perl just to do some text processing?

That mess of redundant code, bad practices and unfit toolset would only prove our point further.

If you don't feel comfortable in Bash, maybe *nix ain't for you.

This violates my number one rule of shell scripting:

Don't use symmetrical quotes

So you never use 'single' and "double" quotes?

This is a start, but I'd like all my unix utils to input and output structured data instead of walls of text

https://github.com/rcrowley/json.sh

Powershell does! :)

Yup. Posh ended my habit of installing cygwin, and now I cringe when I have to do it the "old-fashioned" way on a Linux box.

Powershell is more verbose, but much more expressive.

jsawk : https://github.com/micha/jsawk

and a slimmer tool inspired by jsawk (in node) https://gist.github.com/1053262

There are code modifications to bash to use PCRE and MySQL contexts. I mean, these modifications are great and all... but I have yet to see the real point behind them until they come standard in any distribution. Before that, they are just play-toys since they are non-standard... IMHO.

Don't waste your time on this, as the author said:

"Note: This is just a fun hack. You may want to consider using mature languages like Ruby or Perl to solve actual real life problems. Oh who am I kidding, I use whitespace and brainfuck every day."

I'd rather see an extensible quoting syntax a la Postgres' pl/pgSQL, e.g. ``j` {"a": 1}``. When you want to support YAML: ``y` {a: 1}``.

Things like this can be very useful on servers where its not possible to install any Json parsing libraries.

What do you mean? Using Python you can just place the library direction in the working directory of your script. I don't see how this library is different from any other library for other languages.

Or XML! https://gist.github.com/1074339

At that point why not just write your scripts in Javascript, and run them with nodejs?

absolutely... Im imagining a 100% javascript shell. 'jash' ?

node.js is great for all those perl-y 1 to 10 liners. It should be the shell.

Maybe not a shell, but for simple scripts you can save your js code with the following at the top:

  #!/usr/bin/env node

Then save the file as "myscript" and run it as any other executable. Node already has a repl for experimentation.

yes, and I do that often.

My point is that Javascript + Node would make the basis for a nice bash replacement.

If you like JavaScript. Java would be better. There is the Bean Shell. That doesn't replace BASH for what it does well. In fact, that's the point of BASH, not to replace traditional languages but to bind their output programs together.

Reply on news.ycombinator.com