Babashka is a fast-starting scripting environment for Clojure | Hacker News Reader

Babashka is a fast-starting scripting environment for Clojure

medium.com

217 pointstosh42 comments

It's this really nice combination of bash, curl, and libraries for JSON, CSV, ZIP files, and almost everything else you could need. Full list at [0]. You can always spawn a shell too.

It starts just as fast as Python too, if not faster, at least on my machine:

   time python3 -c 'import os.path; print(os.path.exists("README.md"))'

   time bb -e '(.exists (new java.io.File "README.md"))'

Python averages around 40-50ms, and bb averages 30-40ms.

[0]: https://book.babashka.org/#built-in-namespaces

> Python averages around 40-50ms, and bb averages 30-40ms.

Sounds like you have a slow harddrive here, rather than measuring the startup of the interpreters.

With hyperfine:

    $ hyperfine --warmup=100 --runs=1000 'python3 -c "import os.path; print(os.path.exists('"'README.md'"'))"' 'bb -e '"'(.exists (new java.io.File "'"README.md"'"))'"''
    
    Benchmark 1: python3 -c "import os.path; print(os.path.exists('README.md'))"
      Time (mean ± σ):       5.3 ms ±   0.7 ms    [User: 4.9 ms, System: 1.2 ms]
      Range (min … max):     4.5 ms …   9.8 ms    1000 runs
    
    Benchmark 2: bb -e '(.exists (new java.io.File "README.md"))'
      Time (mean ± σ):       6.0 ms ±   2.5 ms    [User: 2.5 ms, System: 4.3 ms]
      Range (min … max):     0.4 ms …  10.6 ms    1000 runs
    
    Summary
      'python3 -c "import os.path; print(os.path.exists('README.md'))"' ran
        1.12 ± 0.49 times faster than 'bb -e '(.exists (new java.io.File "README.md"))''

Running on the following "not absolutely shabby" specs:

> Intel i7-1185G7, 6.0.11-arch, Toshiba NVMe SSD (KBG40ZNS256G)

Seems Python has more consistent startup performance (`4.5 ms … 9.8 ms`) while Babashka can startup faster (`0.4 ms … 10.6 ms`) but on average, Python startups faster.

bb is 1.32 ± 0.08 times faster on my computer (m1 mbp).

This is a particularly joyful piece of open source software. After seeing Michiel Borkent -- its author and writer of OP -- present on it at a meetup, I wrote a tidy overview of babashka (bb) which you can find here:

https://amontalenti.com/2020/07/11/babashka

The positivity in the comments here makes me want to learn Clojure. Apparently it makes you happy.

It absolutely makes you happy. Be beware, finding a language that makes you too happy, can lead to you not wanting to program in anything else anymore. You're better off being in the middle of grumpy and happy, like a normal programmer.

Don’t worry, folks! Clojure will make you happy, but even with fast perf you may find yourself wanting static types or even just dynamic runtime result/option types. Of course it’s a lisp with macros, and a general purpose language, so you can have all of that too. But there’s no shortage of green in other pastures ;)

this is why i avoid common lisp now. for years i hated programming until i finally decided CL was dead to me. Now i happily write Python for a living and don't care it sucks relative to CL.

Don't learn a Lisp. For one, you'll end up in endless arguments whether your Lisp is a proper Lisp, and two, everything else will look ludicrously constraining for no good reason.

Lisp-1s are bastard children of the true Lisp-2s /s

this is good advice. its a trap. you live in a world where lisp isn't going to be accepted, for better or worse, lisp isn't going to happen.

FWIW, I work fulltime on a Clojure codebase. There are other companies who also use it (for a more well-known example Metabase). Somehow Clojure is a business-friendly Lisp, probably because of the JVM and all the enterprisey stuff that's available for it.

Yup, the JVM is the number one reason why I avoid it like the plague. Which is too bad because the language is super slick, I like it better than Racket and the other schemes.

Have you ever worked/tried to work with a lisp professionally? I don't think what you're saying is true, at least today. I've worked with many Clojure companies for the last years, for them lisp has already happen and is still happening.

Main drawback of using a lisp for professional development is that it's really hard to go back to anything else afterwards. "Why use Assembly if you have C?" (unless you have a really good use case for it, of course), that's a bit the feeling you get.

I'm doing Advent of Code in clojure for the second year. It's frustrating in some ways (learning the string-parsing to process the inputs takes longer to learn in clojure than to simply execute in C++, C#, or JS, but that's because I know those other languages much better).

But the pleasure of iterating towards the solution and building and quickly testing each of the constituents is joy-inducing.

That's funny because I've found the exact opposite. Most everything seems to use roughly the same utilities, but Clojure lets me move more quickly in getting the inputs parsed correctly.

For example, I often start with something like `(slurp "inputs/day-2")` and keep wrapping that until I get a structure I'm happy with. Then I split it into a few fns so I can keep testing with the smaller input

I'm at the stage of using clojure for a total of perhaps 50 hours, 40 of which has been in AoC 2021 or 2022 and no prior JVM experience to draw on the java library ecosystem fluidly (which is a massive strength of clojure, but one that I'm poorly equipped to use).

My solution times so far have scaled pretty heavily with "how long did I futz around with parsing the input?" I did well on days 3, 4, and 6, because the parsing was so trivial and it was just a quick function composition. Day 5 I spend entirely too much time parsing the input (In retrospect, I probably should have just edited it into an initial data structure in emacs.)

Honestly, I think a good chunk of the challenge with a lot of Advent of Code puzzles comes from parsing the input. That’s where I end up spending a lot of my time too, especially when it needs something more than parsing each line individually or reducing by lines.

I think this is one of my favourite pieces of tech in the past five years tbh.

I still use bash for short <5 line scripts but everything else is bb (though I’ve started looking into nbb because you can use node libs like ink which seems pretty cool)

And repl integration with neovim and conjure is great!

I've been looking for a good excuse to use Clojure, small shell scripts sounds like a great low-risk way to do that.

I used this for my back up system: https://github.com/eamonnsullivan/backup-scripts

The server runs on a Raspberry Pi with a 1-2TB USB disk attached.

babashka is so good it's mind-boggling.

super-easy shell interop + instant nREPL = replaces any complex bash script, driven from emacs!

what's more, you can namespace your scripts and import them. functionalizing code makes scripts easy to refactor, or even trivially move into a full Clojure backend or ClojureScript frontend. With squint-cljs you can even generate self-contained JavaScript directly from babashka code.

All our cron jobs and scripts with non trivial logic are in babashka now. It has been a joyful experiences

I've been using it for quick web scrapping scripts and it's really nice.

What libraries do you use? I do most of my scraping in Python using beautifulsoup.

Not OP but I use Reaver with good results. It supports all of JSoup's selectors, and makes it very clean to extract data from HTML.

The documentation is a little lacking though, I had to look up other examples on GitHub to figure out how to use all the features.

https://github.com/mischov/reaver

Babashka doesn't have a built-in HTML parsing library but it supports it through pods:

https://github.com/babashka/pod-registry

Pods can be written in any language and they can expose functions to babashka by implementing a protocol.

One pod exposing HTML parsing is:

https://github.com/retrogradeorbit/bootleg

Here is an example of how to use that:

https://github.com/babashka/pod-registry/blob/master/example...

As mentioned by the one and only Borkdude, bootleg is a nice option for this.

It includes the Hickory library: https://github.com/clj-commons/hickory

I'm a previous BeautifulSoup user and have found the combination of (1) having the scraped data presented in plain Clojure data structures, and (2) Hickory's built in selectors, to be a very nice experience.

Happy scraping!

I plan to port my scraping framework (Skyscraper, https://github.com/nathell/skyscraper) to babashka one day. I’m not sure how easy it will be, though, since it uses core.async (which I believe bb has limited support for) and SQLite via clojure.java.jdbc.

Core.async is listed as “built-in” in the Babashka Toolbox (https://babashka.org/toolbox). Might be worth checking if the compatibility has improved.

And bb supports Honey SQL and SQL pods (https://github.com/babashka/babashka-sql-pods) - so you might be already compatible!

As other people have said Bootleg + Hickory.

Here is an, admitedly not very clean, example that grabs stream urls from hltv.org:

https://github.com/TimDeve/.dotfiles/blob/master/scripts/gen...

Also a basic RSS reader using the clojure XML lib:

https://github.com/TimDeve/.dotfiles/blob/master/scripts/gen...

Thanks everyone, all great pointers. I'll do my next scrape in Babashka!

BB is awesome - it powers our RabbitMQ management scripts, we have written a sidecar process to scrape metrics from ECS and send them to vector.dev over UDP and a bunch of other things.

For personal stuff I use it in a lot of scripts used by Alfred.app including quick-fill for Things with date parsing, emoji completion, etc

I personally stopped using it since both shellcheck and jq fill their own niches way too nicely “unfortunately”…

None the less, a lovely tool and the couple of scripts I wrote are mostly still in use and well maintained for a reason ;) Clojure is just fantastic and so is the speed of the GraalVM.

Babashka is great! I've used it for doing some munging of csv and xml files.

Is there something similar for Common Lisp?

I think SBCL starts up fast enough as-is, no? If you want a batteries included version with libraries, just load them and save a new base image with SAVE-LISP-AND-DIE

No, we have to build a binary, which starts up super quickly.

I began to put together a "distribution" of useful CL libraries for everyday tasks: https://github.com/ciel-lang/CIEL/ It comes as:

- a lisp core, which you can use in your editor setup instead of sbcl or ccl, the advantage is that it loads instantly with all these libraries built-in (instead of quickloading all of them when needed)

- a readline REPL, with completion, syntax highlighting and some goodies (a shell passthrough, quick doc lookup…), albeit it needs more love

- a lisp library which you can quickload like the rest, it just comes with a bunch of dependencies.

- I want to add scripting capacity too.

It ships libraries for: HTTP requests, file formats handling (JSON, XML, CSV, TOML…), shell utils, file utils, threads, a nice hash-table notation, DB access, plotting, strings, other language extensions etc.

It is not finished but I am doog-fooding it for clients' projects.

Thank you very much!

So the scope of it is like Janet-lang, right?

Howdy, I just added scripting support.

    $ ciel myscript.lisp

where we can use all the available libraries, it works instantly.

It's brand new, not many options right now ;)

> like janet-lang

I don't think so, since Janet advertises itself as an embedable Lisp like Lua or Guile (and ECL). But yes, in the sense it comes with batteries included. Like Babashka, I guess.

Thanks for your initial question!

---

So, in CIEL scripts right now, we can make HTTP requests (dex:get or http:get), parse JSON, CSV etc, run shell commands (cmd:cmd, the "!" shortcut in the CIEL shell), or even

    (defalias ! #'cmd:cmd)
    (! "pwd")

and more.

https://github.com/ciel-lang/CIEL/

and just added running the file, with an appropriate shebang:

    $ ./myscript.lisp

BTW, Roswell makes it easier to run scripts: https://github.com/roswell/roswell/wiki/Roswell-as-a-Scripti...

It is also a tool to install various CL implementations, and to install software.

It doesn't come with a choice of built-in libraries.

Reply on news.ycombinator.com