Shake: Every Program Can Be a Clojure Function | Hacker News Reader

Shake: Every Program Can Be a Clojure Function

sunng.info

79 pointssunng25 comments

Shake draws inspiration from Python's sh. Which looks to me like a port of http://perldoc.perl.org/Shell.html. However after experience, that original version now comes with important warnings about things like metacharacter quoting not being able to be done in a portable and safe way.

It is easy to get things 95% right, and I am sure that Shake does that. But think twice before using it for production programs, particularly if there is any possibility of it encountering untrusted input.

> We will have a beautiful DSL so you don’t have to quote arguments as string...

If i'm using Perl's Shell.pm (as mentioned elsewhere here by btilly) and wanted to avoid quoting arguments then I could do:

  use 5.016;
  use warnings;
  use Shell ();

  use PerlX::QuoteOperator ls => {
    -emulate => 'qq',
    -with    => sub ($) { Shell::ls($_[0]) },
  }; 

  use PerlX::QuoteOperator uname => {
    -emulate => 'qq',
    -with    => sub ($) { Shell::uname($_[0]) },
  }; 

  use PerlX::QuoteOperator ip => {
    -emulate => 'qq',
    -with    => sub ($) { Shell::ip($_[0]) },
  }; 

  # and then...

  ls();
  uname(-a);
  ip(-4 addr);

But this seems to be an overkill because I could just use Perl's backticks (http://perldoc.perl.org/perlop.html#Quote-Like-Operators)...

  `ls`;
  `uname -a`;
  `ip -4 addr`;

And it works with variables...

  my $x = "/usr/local/";
  my $files = `ls -l $x`;

and piping...

  my $perl_files = `ls | grep .pl`;

Shell.pm also works with variables & piping but I think that backticks are probably Perl's best DSL for dealing (easily) with shell stuff :)

This is a really bad idea. It's not even close to useful right now. For instance, you can't use variables in your shell invocation, so this doesn't work:

    (let [x "/usr/local"] (ls -l x))

And it clobbers so many things in the core namespace when 'use'd.

Good luck.

> And it clobbers so many things in the core namespace when 'use'd.

Yes the ...it indexes all the executables in your path... is different to the approach used by Perl's Shell.pm and I think Python sh.

These simply catch any undefined function calls and attempts to execute them in the shell (found in your $PATH).

  use 5.016;
  use warnings;
  use Shell;
  say cat("~/.vimrc");              # runs /bin/cat
  say incorrect_spelled_command();  # ERROR can't find in $PATH

However nothing is clobbered here:

  sub cat { "This is my cat()!" }

  use Shell;
  say cat('~/.vimrc');  # => "This is my cat()!"

The best approach is to not use the lookup dispatch and instead tell it what executable you want to wrap...

  use Shell qw(cat);

Now incorrect_spelled_execute is never passed to Shell.pm and in fact will give a compilation error (Undefined subroutine). And any cat() subroutine will be overwritten but it does give redefined warning.

All these efforts to provide pretty & convenient interfaces to arbitrary executables strike me as being useful only for people who want a powerful REPL as their command-line prompt and/or would rather use Clojure or Python (or whatever other language) for mundane tasks that would otherwise normally be done with throwaway shell scripts. Outside of these limited use cases, I'm not sure this sort of thing is a good idea.

Creating this was probably fun and a good learning experience, but I wouldn't recommend trying to push it beyond that point. The problem Shake solves doesn't call for the sort of DSL that uses macros to make Clojure code act like something other than Clojure code.

I'd rather have something like https://github.com/clojure/tools.cli but in reverse: a library for building, manipulating, and running command lines and interacting with the results ... maybe something that supports setting up mappings so that external programs can be exposed as Clojure fns with "Clojuresque" arguments "--rather" "than" "--arrays-of" "strings".

A Scsh port to clojure would be nice. (Scheme -> Clojure is not such a stretch).

That's both wonderful and scary. I think the UI of it needs work -- if you want to treat all shell commands as functions, they imho should just return their outputs by default. (Potentially as a lazy list of lines?)

Also, I think that path should not be passed in as an environment variable, but in Clojure somehow. There's no reason for a program to pollute a global namespace with it's internal details.

Agreed, namespace clashes are an obvious problem. "My program that worked last week now crashes? Oh, it's because I apt-get installed something completely unrelated."

Well, (use :as) fixes that one.

Namespaces are a honking great idea, let's have more of those!

Also, I think I just realized an another problem -- if I understood the lib correctly, it scans through the PATH once at require time. This is bad, because Clojure is often deployed in places where processes run for very long times. Even things as immutable as tools in PATH can change when given long enough time.

There's no (use :as), only require. (require [shake.core :as sh])

That's a step in the right direction, but it still triggers tons of warnings replacing vars in clojure.core.

use does accept an :as option:

    => (use '[clojure.set :as set])
    => set/intersection
    #<set$intersection clojure.set$intersection@58ca40be>

Prior to the addition of :refer to require, this was often paired with an :only (...) in order to alias a namespace as well as refer in some limited set of vars if desired, all in one declaration.

Huh, I checked the docs before I commented. It is indistinguishable from require :as, thought, right?

As far as I know. Honestly, by use/require/refer-fu is not very strong, I always just use the ns macro.

Yes, the :as option is the same whether you use it with require or use.

I'm also looking for better ways to index path. If I can pass some options in (use) and initialize with options, it will be better.

I really think "not indexing path" is the better choice. Instead of (uname -a), why not just export a single symbol and do (shake uname -a)? (or even just (sh uname -a)?) Then find the correct program to run at invocation time. It's more clear about what happens, and allows much simpler implementations with much less weird failure modes.

Sounds very much like pallet's stevedore.

http://palletops.com/doc/reference/script/ https://github.com/pallet/stevedore

This was already pioneered by Olin Shivers' SCSH (Scheme Shell). See the process notation chapter in the SCSH Reference Manual: http://www.scsh.net/docu/html/man-Z-H-3.html#node_chap_2. Wish the SCSH project would come alive again: https://github.com/scheme/scsh.

That's great. Next step is implementing piping :-)

actually, pallet's library stevedore offers the same kind of functionality and has had piping for a good while (as well as other typical constructs):

See: http://hugoduncan.org/post/2010/shell_scripting_in_clojure_w...

The project lives at: https://github.com/pallet/stevedore and is heavily used in pallet, clojure's configuration management and cloud control library

Not to be confused with http://stevedore.readthedocs.org/en/latest/index.html which is part of the virtualenv (or virtualenvwrapper?) installation for Python.

This is actually pretty neat, specifically the part about indexing the executables available in your path and turning them into macros. I was underwhelmed at first because I thought you manually implemented each program.

I can't really imagine a use case for this for me (bash! But with parens!) but still very neat idea and cool execution.

A function? Really? Without side effects?)

You can set environment variable SHAKE_PATH to specify path that you want to initialize.

Reply on news.ycombinator.com