Clojure builds as an amalgamation of orthogonal parts | Hacker News Reader

Clojure builds as an amalgamation of orthogonal parts

blog.fogus.me

122 pointsfogus96 comments

It's nice to have complex build/test setups with deep directory structures, I'm really happy this stuff exists and is being enhanced, but the flexibility kinda adds a barrier to getting started. It's also important to be able to just drop into a REPL, load some libraries and go.

I think the coolest thing in the pipeline is going to be `add-libs`: https://insideclojure.org/2018/05/04/add-lib/

I made a little orgmode literate demo that is selfcontained (no deps.edn) and even produces some inline SVG https://geokon-gh.github.io/literate-clojure.html

I hope this gets added into `core` and single file Clojure programs become a norm. Right now you still need to add tools.deps.alpha into your default user deps.ends for things to work. If this stuff gets into core you could finally send someone a file/gist and they could run it directly. I think that'd really improve the Clojure ecosystem. You can imagine sharing one-file issues/demos/examples and people don't need to reproduce your local setup or clone a whole repo

> I'm really happy this stuff exists and is being enhanced, but the flexibility kinda adds a barrier to getting started

Clojure (seems to me at least) has always preferred to cater to professionals who want to learn languages, tools and concepts deeply rather than being quick to learn. So naturally, the ecosystem tends to lean towards simple tools you use as building blocks rather than one-size-fits-them-all ala Ruby on Rails. This does make it harder to get started with Clojure, but once you've taken the time you to learn everything, it really pays off.

The tools pay off for sure but even once you know how to use deps.edn and company, I'd argue you still really need quick one-file programs.

It's not just about education (though that's important as well). I think the top two use-cases are scripts and issues.

Issues, .. well you want to be a nice dev and post issues with some sample code that reproduces the problem.. but you don't want to have make a whole repo (and then have it floating around on your github indefinitely?) just to show an issue/problem that takes 20 lines of code. So then most people end up copying the problematic code into the github issue text box.. and... it sorta kinda works? but not really. Neither you nor the library maintainer can actually copy and run the code now, so you both end up squinting at it and trying to guess what went wrong. It's just a mess. Even if you make a repo, then how do you get feedback? Other people need to fork and upload their own versions of your demo repo? It's just too heavy handed.

For scripts and quick hacks you want to bang out code quickly. Open up a new file and write .. slurp some files, massage some data, look at some values .. maybe spit out a plot etc. And you want to be able to quickly look over them 2 months later and see what the hell you did. You want to be able to send and share them easily. It really needs to be all in one file otherwise it's just too much of a hassle.

I think the final killer features of `add-libs` that I'm appreciating more as I use it - is that since you can add libraries right in the REPL dynamically you never need to modify your deps.edn and reboot CIDER! It seems minor, but restarting CIDER just always sucks.. and at least for me it breaks the "flow". You loose all the state you had (if you're poking around you can have lots of little variables around) and you loose all your command history. You can only modify your workflow so much to accommodate the inevitable occasional CIDER reboots.

Babashka fills the scripts / one-file programs niche rather well. It comes with many built-in libraries[0], add-libs equivalent, and very fast startup time

[0] https://book.babashka.org/#libraries

gists are git repos and can contain multiple files. Since clj supports git deps, you've been able to use them as deps for a long time.

Example gist: https://gist.github.com/puredanger/cbcdd9f54deb7fb7309fdd632...

Run it: clj -Sdeps '{:deps {a/gist {:git/url "https://gist.github.com/puredanger/cbcdd9f54deb7fb7309fdd632..." :sha "116ac594c922be6ed9d3366f5ac89ed2b0a082f5"}}}' -X foo/bar :name '"Alex"'

that's pretty cool! I guess gists are a bit fancier than a normal pastebin. They don't really advertise it and they don't even display the sha hashs. I didn't realize that :)

and thank you for the demo. With the`:paths ["."]` you flatten the project and minimize the example size. It would make demos look much better

In the end it's not quite as ergonomic as using `add-libs`, but it something that works today. Hope`add-libs` makes it into core some day. Having one-file programs would really take away the remaining friction of sharing code.

PS: thank you for your help on ask.clojure.org the other day. I got your book and am looking forward to reading it

add-libs is definitely still on the future list!

Looking forward :) I just know getting things in core is difficult b/c understandably you wanna keep most things out in utils libs.

But if you wanna have launchable scripts/code-snippets then having to ask people to append -Sdeps blahblah complicates things a bit

btw, I had a question about dependencies resolution that's always been a bit mysterious for me

from: https://insideclojure.org/2018/05/04/add-lib/ "Additionally, this code is not just blindly adding dependencies you ask for but is actually considering them in the context of the dependencies already on your classpath. So if for example, you already had a version of org.clojure/data.priority-map above, it would do the dependency resolution in the context of that version and not resolve to or add a new version to the classpath."

If you have libraries A and B and they depends on library C. Say A has it tagged at version v1.234 and B has it tagged at v1.345. From the doc's glossary it sounds like it will just get the latest version and hope nothing breaks. https://clojure.org/reference/deps_and_cli#_glossary

I haven't had it happened, but I could imagine a scenario where you add library B and library A starts to misbehave. And if I understand correctly, there is no way to have multiple versions of a library in the path. Is that right?

I know in C++ world this is a big unspoken issue. Package managers usually just mash things with the latest version and hope it works, while some build systems (like Hunter) will modify symbols so you can link to two copies.

What all the critics miss is that Lein is "Easy" but tools.deps, tools.build are "Simple". Rich wants all of clojure to be built from simple orthogonal parts that compose together. Ideally a multi-purpose tool like leiningen should be built on top of those simple parts. Other tools can re-use those parts in different ways. With Leiningen - its a great tool but it's a complex thing that you take on an all-or-nothing basis.

Theory is all very well, but adding a library to a project is a critical requirement for all projects. A new starter comes over to Clojure from Python and the Deps guide [0] confidently explains too them that they add

{:deps {clojure.java-time/clojure.java-time {:mvn/version "0.3.2"}}}

to their deps.edn and they're good to go. Raising only the questions of:

1. What are the available keys of this map?

2. Why do I have to repeat clojure.java-time twice? Is this arbitrary? When are these different? (turns out it depends, if anyone is trying to figure that out)

3. What is a ":mvn"?

4. Is there a list of libraries I should be consulting to work out what versions mvn supports?

5. Is mvn my only choice here? Are there other repositories?

6. The map is one of Clojure's most flexible data structures because it can take arbitrary keys. Circling back to 1 - What other keys are supported?

Answering 1/6 is particularly interesting because the Deps reference [1] is long, undecipherable and frankly not-very-well written reference material. It starts off as more of a tutorial than a reference and requires the reader to engage with how Clojure programs run rather than how to download a library.

Someone coming over from Python would look at this, recall "pip install library" and then a lot of them would do the sensible thing and give up. Potentially never realising that the correct thing to do is go with leiningen. I'm not even sure if pip has command line flags, I've basically never been exposed to them.

Simple is all very good, I'm sure the people who struggle through to figure out how deps actually works are the better for it, but it would be better if adding a library were easy as well as simple.

[0] https://clojure.org/guides/deps_and_cli#_running_a_repl_and_...

[1] https://clojure.org/reference/deps_and_cli

While I agree that Leiningen is easier to get going with, many of the questions you asked also apply to Leiningen's project file.

I use Leiningen as it's convenient, but I check back on the tools.* ecosystem occasionally to see the progress being made. My suspicion is that tools.* will eventually supersede Leiningen and Boot, but that it will take two to three years at least.

Currently it's easier to use "lein new", but I can imagine a Babashka wrapper around tools.deps and tools.build eventually supplanting this. Being able to write, for example:

    bbb new foobar; cd foobar; bbb add clojure.java.time

to automatically generate and update a deps.edn would be convenient, and potentially with far lower latency than Lein.

These are good points and I'd welcome feedback at https://github.com/clojure/clojure-site/issues

Simple and easy are not mutually exclusive, and simple software is not automatically good software. Other criteria, ease of use, documentation, also matter, especially for a build tool intended to be used by a whole community.

At the moment deps feels more of a 'make your own build tool' kit. Simple, sure, but not easy to use: you need to tell it how to build jars or run tests, which is IMO essential functionality of a build tool.

I don't want to tell my build tool how to achieve these essential things, and risk ending up with a mess of projects that all use different aliases to run different tests runners and different jar building plugins. Build tooling is one place where convention over configuration matters to me, and why I still prefer leiningen.

I'm hoping deps and tools.build will evolve from a 'make your own build tool kit' to a build tool that exposes the kit it was built on, so that most projects don't require anything custom, and be flexible enoug so that one special project can leverage the 'simple orthogonal parts' in the kit.

I see tools.build as a step forwards, but it's not there yet IMO, the user still needs to configure a bunch of stuff to get essential functionality.

I hope the core team will make a build tool on top of the 'make your own build tool kit' that is deps and tools.build, that comes with good defaults and essential features for the common projects, while exposing the 'simple orthogonal parts' in the kit so these can be used for complex projects.

A bit of context, because your post could easily be taken in bad faith:

> Ideally a multi-purpose tool like leiningen should be built on top of those simple parts

tools.build is 2 weeks old according to the announcement.

Leiningen is nearly 12 years old.

I haven't looked into tools.build, so not arguing for or against it.

Disclaimer: Have been doing some work on leiningen from time to time.

Also I first ran into Clojure in 2011, and while I liked the language, I simply loved how leiningen just worked and I was instantly a fan, not even thinking about how it's built, just nice for the users[0]. And I've always hated projects/ecosystems that made you jump through hoops or some arcane syntax and exploded half the times you tried to build. So "easy" is really important.

[0]: https://f5n.org/blog/2011/a-week-of-clojure/

"Easy" is important for someone to start playing around with something, to initially hook the user (the developer in this case).

But for professionals who spend a lot of time with the same tool, "simple" is more important in the long run.

Still not sure I agree here, as long as it's flexible enough to let me get my work done.

The whole thing was "it must be simple, even if the user needs to spend more time learning it" which is only true for people who learn the thing and keep on using it. For us who come back to a project after months (because we switch between tech stacks) it might be just twice the amount of work, if you need to relearn it every time.

It's not just that, or I don't think that is even the primary reason for tools.x.

Spec and tools are being developed to lay a foundation, so one can talk about and build upon the ideas of granular, function level dependencies, change (breakage vs. accretion) and so on. Ultimately this is about how code changes over time.

As a Clojure developer focused on creating correct/working software, I find the "Simple"-ness of my tooling's internal architecture to be a lower priority than it not leaking additional cognitive load onto my dev experience.

It may be the case that lein's internals are a blob of complexity, without boundaries defined in the "right" places. Maybe Clojure Tools' 3 or 4 components have it correct (though I'm skeptical of that, since there's overlap between them -- even moreso now with tools.build).

Conversely, it may be that build tooling is by its nature a messy affair (dealing with dependencies, resources on the filesystem/network, interaction with tools, tests, etc.), and that making those problems go away to the extent technically possible does your community of developers the greatest service. That would explain why talk of CLI and deps invariably has pushback and lein advocacy.

I wouldn't know if lein is internally complex because I've never had to go digging in it beyond the documentation project.clj file, even to do rather complex build definitions. By comparison, I've easily spent man-weeks on getting to the same place with deps.edn, and I would still find even defining the roles of its components difficult if pressed. I also worry that if we hit new Clojure devs with a wall of tooling issues like this, they won't stick around, even if we tell them the underlying theory of their tooling is more sound.

> I also worry that if we hit new Clojure devs with a wall of tooling issues like this, they won't stick around, even if we tell them the underlying theory of their tooling is more sound.

This sounds like a educational problem rather than "lets dumb down our tools to cater to new developers".

Clojure seems to balance "cater to professionals VS being easy to get started with" to the first mentioned segment, something I myself has no problem with.

Clojure projects can start with literally no files (not even a deps.edn). By default, you get Clojure as a dep and src/ as a directory. Run `clj` to get a repl. If you make a function, you can call it with `clj -X my.ns/foo`. If you push it to github, others can consume it immediately by using your git url + sha.

There are lots of other things - they are all additive. Many Clojure libraries and tools don't need a build.clj or tools.build at all.

My experience is that deps.edn is much easier to get started with. With a template from something like clj-new you can get testing, a basic jar building script, etc to get started with and it's all there for you to muck with if needed.

I don’t understand why deps was needed and I’m surprised its authors didn’t foresee the confusion it would create. I’m also confused as hell what I should be using now, and the idea of researching deps seems exhausting. I’ll just stick to cutting and pasting onto my leiningen config I think.

> I don’t understand why deps was needed

First, start here: https://clojure.org/about/rationale

Then Deps and CLI is explained here: https://clojure.org/reference/deps_and_cli

> what I should be using now

As always, depends. Building a project in a rush? Use whatever you're most comfortable with and have the most experience. Want to further your understanding of Clojure the ecosystem? Dig into Deps & co.

> I’ll just stick to cutting and pasting onto my leiningen config I think

Yeah, if that's your general approach to development, use whatever you already know. Once you're ready to understand the tools you depend on, then you can start looking into tools that was built with some more thought behind them, like Deps.

It’s still alpha, and from what I can tell from the links referenced, a severe case of NIH. Hopefully it will be “made simple” and “de-complected” soon though so that I can tell what the point of it is in the presence of Leiningen.

I asked some questions on slack related to this, and in case anyone else was confused like me thinking tools.build is a new framework for managing build tasks, that does not seem the be intent.

tools.build is a helper lib to write programs that make artifacts such as Jars or Uberjars. It competes with things like depstar in that sense.

Tasks are still meant to be managed by deps.edn (tools.deps) and the Clojure CLI by having tasks be Clojure programs that are invokable as either -X, -T or -M, and the idea is you would orchestrate them with another tool like make, a shell command or script, babashka task runner, npm scripts, just etc.

This means that any Clojure program can be a "task". All you need to add a new task to a project is define an alias for it in the project deps.edn.

Now one caveat is depending on the type of program the task is, you might need to call the alias with either -X, -T or -M option:

    clojure -T:alias
    clojure -X:alias
    clojure -M:alias

Ideally all Clojure programs that implement a task move to rely on exec-fns, and thus would be called with -T or -X, which will eventually allow you to chain tasks together where the return of the first task exec-fn will be passed to the next task's exec-fn, so you can compose tasks and run them consecutively from the clojure CLI.

So tools.build is just a lib that you can use to help you write tasks (which are just normal Clojure programs), but it is itself a set of exec-fns tasks which you can use directly as an alias as well:

    clojure -T:build-api copy-file :src '"./src/tempo.clj"' :target '"./output/tempo.clj"'

with alias as:

    :build-api {:deps {io.github.clojure/tools.build {:git/tag "v0.1.6" :git/sha "5636e61"}}
                :ns-default clojure.tools.build.api}

I actually have found tools.deps to be pleasurable to use on new projects, though I haven't tried porting anything that previously used leiningen directly. It's simple, it's clear, I guess I find it pretty easy to reason about, but maybe that's because my experience with Lein was more 'copy and paste this huge config' and not really starting from scratch.

Glad they're finally making this stuff first-party. When I messed around and did a Clojure project, the build/dependencies story was the biggest sour note of the experience by far. It was bewildering to have to go set up some third-party tool with a mustache logo before I could follow along with the official tutorial.

I think having a cohesive story for this is table-stakes in a modern programming language.

For as long as I can remember, the instructions from the official docs leads you into a REPL where you can start learning. Not sure what this "official tutorial" you are talking about, but as long as you get a REPL running, you should be able to get started.

I couldn't start writing and compiling my own source files until I went through all these unofficial channels to get stuff set up. I might be misremembering about the actual tutorial itself, but step 2 (do your own project) was literally impossible without the detour.

This is even worse than how it is for Python - which does have a similar problem - because at least in Python's case you can "just run" your scripts up until the point where you need dependency management and packaging. You can't even execute a .clj file (as far as I know) without going outside of the official resources.

I'm not sure how they can make it easier than what it currently is, no need for any unofficial channels. If you follow the guide at https://clojure.org/guides/getting_started to install Clojure, you'll end up with `clj` in your `$PATH`, and then it is as easy as it is with Python, unless I misunderstand what you mean.

   $ clj --version
   Clojure CLI version 1.10.3.849
   
   $ echo "(println \"hello world\")" >> hello-world.clj
   
   $ clj hello-world.clj
   hello world

This was three or four years ago so it's very possibly changed since then (in fact the OP was specifically about this story changing)

Yeah, three or four years ago (before Clojure Tools), you'd have to do something like this:

    $ java -cp clojure.jar clojure.main hello-world.clj

Still, I'd argue it's about the same as Python, at least for the most basic use case, like what you were complaining about in your previous comment.

This is good for clojure, I think. There was a time when lein seemed the preferred tool for clojure projects, but lein seemed to be doing too many different things, and more significantly was not packaged woth clojure. So during that time clojure felt batteries-not-included. With in built tooling, getting into and using clojure should be more easier.

Note to non-Clojurians:

Most people just use Leiningen and don't mess with this stuff.

This post does not paint a good picture of Clojure development tools, but don't get turned off by it.

99% of Clojure projects I've worked on just have a declarative project.clj file and building it is as simple as:

    $ lein uberjar

I will never understand the core team's acute NIH Syndrome and tendency to fragment the community, but I guess Open Source Is Not About Me.

I've tried deps several times, but I keep coming back to leiningen. Can anyone explain the advantage of using deps to me?

What I want from a build tool is: manage dependencies, starting a repl, running tests, and building a jar. Leiningen provides me this: lein repl, lein test, lein uberjar. Simple and exactly what I need, and importantly: i can expect that all leiningen projects use the same commands.

Deps can be made to do all these things, and more, but you need to configure it. Which test runner should I use for tests? Which library should I use to build an uberjar? Since it's all configurable, I risk that every deps project gets it's own special snowflake build setup, and I need to spend time avoiding this.

Until deps gets a standard way for building and testing, i'm sticking to leiningen.

deps is extremely simple, which means very little time spent debugging, and it never breaks

I have never stressed over or debugged Leiningen in my 6 years of Clojure experience.

Not sure what you're on about.

deps can't produce stable builds as it insists dependencies are not ordered. Yet on the classpath they are. That order shouldn't matter, but it very well does.

lein AFAIK produces the same classpath every time.

deps: Maybe put the paths first and then let's gamble what a seq on a map produces...

thanks, didn’t know that, will definitely be annoyed when i hit that someday

That appears to have been true for only about a month: https://groups.google.com/g/clojure/c/WI3ddZRK4Bg/m/RtJVt3IZ...

https://github.com/clojure/tools.deps.alpha/commit/c22ad46c6...

Same. I don't want to spend time learning build tools and configuring builds - I want to Get Shit Done.

Lieningen gets out of my way and lets me do the things that really matter, and I'm not switching till I see some clear advantages of the new tooling.

Exactly. Lein is like maven, declarative and just works in a standard way for 95% of use cases. Deps is like ant, can do anything, super flexible, but you need to start from scratch actually programming your build for every project.

I first used deps when doing a project with Datomic Cloud.

I like deps, but I can’t quite articulate why.

After using it with Datomic Cloud, do you prefer it also for local projects? Is there something that happened with Datomic Cloud configuration that made it “click” for you elsewhere? Did you just have to use it there and that experience informed your preferences?

Data > functions > macros is a core concept of Clojure. Does that come into play here?

These are prompts from a curious outsider’s perspective.

I found an old thread that examines some of the early usage of deps. Data > macros does indeed seem to be a driving force: https://clojureverse.org/t/combining-tools-deps-with-leining...

tools.deps makes better choices when dependency specs conflict, so I would suggest trying https://github.com/RickMoynihan/lein-tools-deps if you otherwise want to use lein, and get the best of both worlds.

By better, I mean:

> Leiningen and Maven, when there is a conflict always pick the version that is closest to the root of the dependency tree; where as tools.deps always picks the newest.

Leinengen vs Maven dependency resolution is pretty distant to my concerns. I may be in the minority there, but I don’t think I am. I guess the thing I’m struggling to understand in the build tool differences is:

Leinengen’s (defproject) takes a series of keys and values. It is declarative. You can probably get into dependency hell if your config is complex enough, and I’m almost certain it’s doing more than I understand under the hood. (like why doesn’t it just take a map instead of being a macro?)

tools.deps OTOH appears to need both a bunch of top-level definitions in a namespace (instead of a map literal) AND a hand-rolled build.clj file with custom functions to do most of the things that leinengen considers to be boilerplate. This does imply that you have much more flexibility with tools.deps, but it also raises the bar for entry, and it doesn’t appear to be as data-centric.

If I want to do ANYTHING meaningful with Clojure, I need a project environment to start with. I am not confident about how to do this with clj, even after reading https://clojure.org/guides/deps_and_cli#_writing_a_program Maybe I just haven’t found the right guide, or maybe clj wants me to commit more to working REPL-first than my dev environment (VS Code + Calva) has good support for.

What I’m wondering now is, if the gains that tools.deps provide are worth the entry price, does it make sense to have something like “create-react-app” to generate a project directory with sane defaults that can be customized if/when needed, if your project doesn’t fit the 90% case? Does this already exist?

‘dustingetz linked the Simple Made Easy talk. I get the distinction between the two, and I don’t want to swallow the hairball. I’m already sold on the Clojure paradigm.

But I ALSO still want to “get this instantly and start running it in five seconds,” and I don’t understand why I’m not allowed that if I choose tools.deps. I don’t think the two goals are more than accidentally orthogonal.

i agree that it’s possible to have both, but you have to start with simple, and as he said in the talk which we just reread, easy is often relative to your local priors

also those custom hand rolled fns are … a library, which is way better than a build plugin

Thanks. I didn’t even know that leinengen has plugins.

If tools.* is the language maintainers’ preferred method, I think there is value in having a single command to set up a “simple/default Clojure project” in a folder, even if that only cuts the commands down from four to one.

  $ mkdir hello-world
  $ cp deps.edn hello-world
  $ cd hello-world
  $ mkdir src

I don’t understand where that deps.edn comes from in a fresh project. It’d be great to have “clj create hello-world” build one for me in ./hello-world, use my running Clojure version as a dependency (if that’s how “clj” works), create the child src/ (and /test if that’s a best practice) folders, etc. The idea being to generate a local library on-demand with sane boilerplate defaults and almost no knowledge. I can learn later what Else I might need if I can get it up and running quickly first.

Leinengen looks easier to me for my admittedly trivial use cases, because I don’t need a library or plugins at all. And again, I might be in the minority. But if folks keep saying “lein handles 90% and is easier,” and if we can make tools.* handle those same 90% cases trivially, it seems worthwhile.

{:paths ["src"] :deps {org.clojure/clojure {:mvn/version "1.10.0"}}}

that's it

You don't even need that -- those are the defaults for the CLI.

    (! 639)-> mkdir fresh && cd fresh
    (! 640)-> mkdir -p src/myns
    (! 641)-> echo '(ns myns.myfile) (defn foo [{:keys [name]}] (println (str "Hello, " name "!")))' > src/myns/myfile.clj
    (! 642)-> clojure -X myns.myfile/foo :name "filoeleven"
    Hello, filoeleven!

No deps.edn file needed -- until you want to add extra dependencies etc.

If you're using the latest (prerelease) version of the Clojure CLI:

    # this is a one-off step to install the clj-new tool:
    (! 646)-> clojure -Ttools install com.github.seancorfield/clj-new '{:git/tag "v1.1.324"}' :as new
    Installed new
    # now you can run it anywhere to create new app or lib projects:
    (! 647)-> clojure -Tnew app :name myname/myapp
    Generating a project called myapp based on the 'app' template.
    (! 648)-> tree myapp
    myapp
    |____.gitignore
    |____.hgignore
    |____CHANGELOG.md
    |____deps.edn
    |____doc
    | |____intro.md
    |____LICENSE
    |____pom.xml
    |____README.md
    |____resources
    | |____.keep
    |____src
    | |____myname
    | | |____myapp.clj
    |____test
    | |____myname
    | | |____myapp_test.clj

This is already set up with testing and JAR building for you:

    (! 649)-> cd myapp
    (! 650)-> clojure -X:test
    
    Running tests in #{"test"}

    Testing myname.myapp-test

    FAIL in (a-test) (myapp_test.clj:7)
    FIXME, I fail.
    expected: (= 0 1)
      actual: (not (= 0 1))
    
    Ran 1 tests containing 1 assertions.
    1 failures, 0 errors.
    (! 651)-> clojure -X:uberjar
    [main] INFO hf.depstar.pom - Synchronizing pom.xml
    Skipping paths: resources
    [main] INFO hf.depstar.aot - Compiling myname.myapp ...
    [main] INFO hf.depstar.uberjar - Building uber jar: ./myapp.jar
    [main] INFO hf.depstar.uberjar - Processing pom.xml for {net.clojars.myname/myapp {:mvn/version "0.1.0-SNAPSHOT"}}
    (! 652)-> java -jar myapp.jar 
    Hello, World!

You keep saying tools.deps but I think you mostly mean tools.build here.

For the CLI -- which uses tools.deps under the hood (not the brand new tools.build) -- all you need is your deps.edn file: a declarative hash map describing your dependencies and, under aliases, any additional tools you want to use.

The build.clj file is something some folks have already been doing in one form or another for large, complex projects (and would be familiar to anyone using Boot instead of Leiningen).

tools.build provides some common tooling and structure that should help reduce the complexity out there by providing a standard way to do a number of tasks that folks were already doing a different way (yes, including some of the core Leiningen tasks) and to provide a composable, function-based set of tools to help you write your own build scripts.

The bottom line: if Leiningen does what you need, great! Keep using it. If you find Leiningen to be restrictive and not a good fit for your project -- as we did, years ago -- then tools.build might be better. We left Leiningen for Boot many years ago and if we hadn't started to run into bugs/issues with Boot at scale we'd probably still be using it. Instead we switched to the CLI/deps.edn back in 2018 and we've just adopted tools.build which has simplified some parts of the build script we already had.

I haven't used Datomic at all but I do prefer the CLI and deps.edn for all my projects. I really haven't used Leiningen for years at this point -- I mostly only use it now when I'm helping a beginner debug some problem they've run into with Leiningen plugsin.

I do.

I guess it boils down to the fact that I don't find it significantly harder to use than Leiningen, there are great libraries and tools that work well with it (shadow-cljs can source its deps from deps.edn for example), and I wrap the entire thing in a task system anyway (lately, Babashka tasks).

I guess it helps that its the "official" way to do things, but that's not a primary concern for me.

Thanks. I hadn’t even considered Babashka, which I’m quite intrigued by.

It's like we learned nothing from years of suffering with snowflake Ant builds.

Then again, gradle builds seems to be more popular than maven these days.

That is what happens when a new generation has grown without Ant and then a giant behemoth with endless cash for top hardware gets a deal to push Gradle as the official build tool for their mobile OS.

I would think that Gradle is a lot more pleasant to use than Ant...

Not for those that don't suffer from XML allergy, so given that, the outcome is almost the same, with the caveat that Gradles uses a ton of more resources than Ant, as it needs its background daemon sucking up 2GB to pretend being fast.

Hard agree here. As a full time Clojure dev, my experience is that lein "just works". I've also used the Clojure Tools professionally, and have seen the dev team lose countless days of what would be otherwise productive time putzing with it, building infrastructure that lein has had out-of-the-box for years, and trying to make sense out of its inscrutable docs.

Want a project with both Clojure and Java? Want tests that magically integrate with CIDER's test functions? Complex build profiles for different target environments? Lein makes this stuff as painless as possible, whereas these things are possible with Clojure Tools, but have fun wasting time messing with it and doing awkward stuff like inlining strings of eval-ed clojure code in your build file. With Leiningen these are solved problems.

I also find the core team's behavior perplexing. I'd argue the messy tooling they've been producing has probably been net negative value add to their community. Working downstream from this stuff has degraded the dev experience. Thankfully, it can be safely ignored.

I agree. To me tools.build is not a replacement for Lein at all. Lein is more intuitive to most beginners. It's much easier to tell someone to run `lein new some-app` and `lein run` / `lein uberjar`, than to tell them to configure their deps.edn file so they can run some alias with the `-X:...` / `-M` arg to create and run a new project (and who knows which dependency to use / configure to build an uberjar).

For experienced Clojure developers this might seem trivial, but new people (in my experience) are very impatient and want to get started ASAP. Honestly, who can blame them? Why does it have to be so difficult in this day and age?

Another thing I hear Clojure developers say is that beginners can start with Lein, but once you get more experienced with Clojure you can switch to tools.build. Why does a build tool need to be so unusable that you can only use it when you become more familiar with the language? You don't see that with Ruby's Bundler, or Elixir's Mix. They just work, just like Lein does.

> RICH HICKEY: I think that, collectively, we are infatuated with these two notions of easy. We are just so self-involved in these two aspects; it's hurting us tremendously. Right? All we care about is, can I get this instantly and start running it in five seconds? It could be this giant hairball that you got, but all you care is, can you get it.

Simple Made Easy is what attracted me to Clojure, and tools.build, tools.deps, tools.cli are aligned with this mission. Cognitect is doing exactly what they said they would do and have been doing all along.

Re-read the talk here: https://github.com/matthiasn/talk-transcripts/blob/master/Hi...

(PS, a note to Cognitect comms – maybe start here with your next blog post and see if people react differently!)

You forget that on the ground-floor, where programming languages compete for mind-share, easy matters. In fact easy, in terms of getting something working, trumps difficult simplicity any day of the week. Easy is part of what makes an employer want to use your technology.

that is completely true and maybe even clojure’s achilles’ heel, but you can’t come here and say they didn’t tell you

> In fact easy, in terms of getting something working, trumps difficult simplicity any day of the week. Easy is part of what makes an employer want to use your technology.

The whole point of the talk is that choosing "easy" (as in easy to get started) solutions while ignoring the complexity will be harder to maintain in the long run over choosing the "simple" solution that takes a bit more time to set up. Personally, this tracks pretty closely with my experience.

(Relevant slide from the talk: https://res.infoq.com/presentations/Simple-Made-Easy/en/slid...)

Bit bold to assume that the people making these decisions "forgot" anything. Design's about trade-offs.

The claim here is that the thing that makes your language easier to pick up initially nonetheless hobbles it for long-term use - refactoring, adding features. There is a business case for making the latter easy at the expense of the former.

Code can be easy or hard to write and/or change at various levels of complexity, and it's hard to solve for everything when designing a language.

To get started with clj, you need 0 files. From a project directory, you can just type `clj` and get a REPL that includes Clojure and source files in src/. You can add new source files and immediately invoke them with `clj -X`. You can push that project to github and others can immediately use it from another project using a github url/sha. This is far "easier" than leiningen for getting started. The comparative experience there requires creating a project.clj with a bunch of keys, building a jar, getting an account and keys on clojars to deploy it, etc.

There's lots of other things you can do with clj - these are all additive. At some point, you might (possibly) need a build script. When you do, it's written in Clojure using a pretty straightforward set of functions and you can copy/paste in a starting point. You can grow that build file forever.

Rich can try to redefine the English language all he wants, but he'll find it a lot more difficult than redefining Clojure.

The fact is, the solutions the core team have been putting out for the last several years have been neither simple or easy, and in fact tend towards a weird and arcane complexity in service of problems no one seemed to have.

I remain baffled by Spec, by the clj tool, by deps, and now this. At every step it seems as if the team lives in their own world, and not in ours, and each new tool is as cryptic and convoluted as the last.

They frequently don't even seem to encourage code that is consistently idiomatic to how the rest of us even write Clojure. Who else was even using namespaced keywords for anything until Spec decided to not only use them, but link the entire mechanism to a mutable namespace registry? How is that "functional" in any way?

> and tools.build, tools.deps, tools.cli are aligned with this mission.

No, they are neither simple nor easy. They make new project setup un-necessarily hard, they make builds un-necessarily hard and complicate stuff for both newbies and intermediate programmers. Only experts with tools.* may be comfortable with this and I have seen these experts actually copy sample template `deps.edn` for new projects and use long complicated aliases.

For both simplicity and ease-of-use this should be done by a tool and not a human.

When I recommended some friends to Clojure, they stopped after getting stuck in deps.edn. Clojure setup feels more complicated than C++/Java/Kotlin/Python project setup nowadays.

I wish Clojure would use Rust's Cargo for inspiration.

`cargo new --bin projName` and open the folder in your editor/IDE of your choice.

We migrated from Leiningen to Boot and then to the CLI/deps.edn. Leiningen was too rigid and too much ceremony to write new tasks for. Boot was great from the "writing new tasks" point of view but the fileset abstraction and the pod pooling were both pain points for us. We've been very happy with the CLI and deps.edn and we're already using tools.build at work to streamline our (Clojure) build script.

We have a fairly large codebase (113K lines) in a monorepo with over a dozen artifacts built from about fifty subprojects. Leiningen really wasn't a good fit for us so we're very glad of alternatives.

Thank you for the perspective (and also for clj-new — it feels like it should be built into Clojure’s own tooling).

Now that tools.build is here, I think clj-new can be substantially simplified (copy-dir with text replacement is most of what clj-new does for built-in templates) and when add-libs gets to the main branch of t.d.a or Clojure itself, that takes care of a lot of the rest of what clj-new does.

> I'd argue the messy tooling they've been producing has probably been net negative value add to their community.

I feel the same way. It has definitely hurt the ecosystem.

Every Clojure IDE, for example, has to support a new deps/build system that is still a WIP and doesn't come with batteries.

Start a REPL; connect to it. That's identical for every single editor/IDE. The problem is that editor plugin maintainers seem reluctant to encourage developers to start REPLs manually so they have to deal with the whole issue of "how to start a REPL" -- which should be up to the developer, not the tool.

I run REPLs for weeks, sometimes months, so I don't want my editor to start my REPL since my editor is going to get restarted a lot more often than my REPLs!

Leiningen is over 10 years old and has tons of pain points. For example, I bet you're not using ClojureScript, which is a mess of plugin debugging in leiningen. The goal of tools.build is to let you code your builds as simple functions that run at the REPL, which is awesome. No more finding misplaced keys in soup like https://github.com/technomancy/leiningen/blob/master/sample.... . If leiningen works for you, great, keep using it!

What pain points are those? Last I checked, doing cljs on Clojure Tools wasn't even possible. If that's still the case, then that's not a valid comparison (complex vs impossible).

Also, that's the master version of project.clj, which is a superset of all possible features lein supports. Nobody's project.clj file looks like that, though I suspect you know this. Should we compare that complexity to a monster build.clj and this master deps.edn:

https://github.com/seancorfield/dot-clojure/blob/master/deps...

Lein is also a singular tool vs. a murky interaction between a suite of tools (tools.deps with deps.edn, Clojure CLI, and tools.build).

To be clear, you are pointing to this as an equally contrived example as what ‘dustingetz linked to, correct?

Correct.

That isn't a project deps.edn file though -- it's a compendium of tools that are available for the CLI/deps.edn and I put it together as a working example so folks coming to the CLI/deps.edn stuff could see the broad range of tools available and could copy'n'paste any useful bits they want.

The README explains that and even links to a better-documented and more carefully curated set of aliases that folks might want to use instead (the Practicalli repo).

In reality, I use very, very of those aliases from that dot-clojure file, because the projects I work with already have the ones I need for working on those projects. That can be as simple as a :test alias or it can be up to a dozen aliases used in variations combinations for a variety of tasks.

You can certainly use the CLI and deps.edn for ClojureScript, by the way -- check out Figwheel Main which uses deps.edn and provides a nice workflow.

That should read "I use very, very FEW of those aliases"

Thanks for the reply. While you're here, I'll say that your resources for the CLI stuff made it possible to use those tools at all (for my current Clojure team) several years ago. depstar also is a great resource.

We had to scour the internet and engage in weeks of teeth grinding to get CLI/deps to do all the things our project needed.

Thank you. As someone who follows the (b)leading edge of Clojure and its tooling very closely -- and has run alphas of Clojure itself in production since 2011 -- it's very hard for me now to put myself in the shoes of someone coming new to this tooling (unfortunately).

I think perhaps the biggest issue for most people is discoverability: how do I learn about the tools I need to get "task X" done with the Clojure CLI, deps.edn, and associated "stuff".

Even though I created clj-new (and boot-new, on which it is based, which in turn was based on lein's new), I pretty much never use it myself, except for quick examples to test stuff and/or help beginners with. I always start a project from scratch with an empty deps.edn and an initial src/myns/myfile.clj or similar.

I use depstar heavily but I strongly suspect that it will ultimately go away as tools.build matures to the point where all the special cases that depstar handles today are folded into t.b.api/jar and t.b.api/uber -- I don't want to maintain something that duplicates what the core team have poured so much time, effort, and careful thought into!

I'm already contemplating a "reboot" of clj-new as a very simple wrapper around t.b.api/copy-dir with some simple templates -- since that's most of what clj-new does (even the more complex template-running stuff is really only a slight augmentation of t.b.api's create-basis + java-command + process).

> Last I checked, doing cljs on Clojure Tools wasn't even possible.

I'm not sure what this is referring to here. The "Getting Started" guide for ClojureScript uses deps.edn and the clojure CLI to introduce ClojureScript. You can certainly use figwheel or shadow-cljs with clojure tools.deps and deps.edn. And since the new build.tools allows just running a program, you can of course build clojurescript from a `build.clj` build script. I cannot think of a step in this process that isn't amenable or simplified by the use of the clojure cli, tools.deps, and now build.tools.

That file says at the top:

  ;; This is an annotated reference of the options that may be set in a
  ;; project.clj file. It is fairly contrived in order to cover lots of
  ;; different options; it shouldn't be considered a representative
  ;; configuration.

The comment from ‘bm3719 makes tools.build sound like a less useful solution when compared to the practicality of leinengen, and your reply only has ClojureScript in its favor if I trust that the linked sample project.clj is indeed not representative (which matches my limited experience with my own “useful toy” projects and the other projects I have browsed). “awkward stuff like inlining strings of eval-ed clojure code in your build file” sounds really bad, but that could be equally contrived for all I know.

I also don’t understand the value of running builds from the REPL. What kind of developer process makes that a useful thing to have? What am I missing?

These are genuine questions, not advocacy for one thing or the other. I want to work professionally with Clojure(Script) enough that it is a prerequisite for me to consider changing jobs. I’ve only used lein thus far, so I don’t have enough experience with either tool to share a meaningful opinion here. I am extremely curious about the state of things though.

Running builds from the repl let’s you skip the startup cost associated with Clojure. It’s faster if you need to run more than build command, or one build command several times.

Lein is easy to get started with, and for simple projects does everything you need. tools.build is endlessly flexible. For some projects, tools.build/tools.deps was much easier for me to use than leiningen (easier to just write clojure code directly than figuring out the whole lein plugin stuff). Use whatever works for you.

Ya, this accords with my thoughts when I first saw this bubble up, I don't understand the benefit over just using lein, which is very easy and straightforward (for my use case). I thought maybe this new approach is needed by people with complicated builds? My own builds have been quite simple so far, but I'm new to Clojure.

I have worked on pretty large Clojure projects, and found Leiningen to be perfectly adequate.

Although I prefer Lein at this point one of its disadvantages not mentioned here is that it requires its own JVM process which means you're running 2 JVMs during development.

but probably good and bad. pretty tough for one namespace to bleed into another due to a bug or incorrect config.

bad, obv, because all the RAM the JVM needs.

I like Clojure but it is not my primary development language so I am happy to stick with lein. As others here have said, lein is easy to use and gets the job done of creating new projects from a variety of templates, building, and deploying.

Very well written post. Simplicity is a good thing!

Given the general tone of comments here, I thought it worth pointing out the advantages of using deps.edn instead of a Leiningen project:

- REPLs start up very slightly quicker

I don't understand why people keep complaining about slow REPL startup time. How many times do you really restart your REPL? In my day job, I usually start up the REPL once and then close it when I'm done during the day, or just fold up the laptop which suspends the REPL and then it's ready to go when I unsuspend.

Only time I can think of when I need to restart the REPL is when I add a new dependency, but that doesn't happen so often.

So why you need fast REPL startup really?

I also go without restaring the repl for weeks, but there are plenty of situations where you need to restart the repl: adding libraries, servers binding to ports, making sure you have no temporary edits, etc, etc.

The whole 'reloaded workflow' is a workaround to the slow restart IMGO. It works well, but there is a learning curve, especially coming from languages with fast edit-compile-run cycles such as go.

I've also worked with scheme, and having a repl up and running in milliseconds is really nice.

> I also go without restaring the repl for weeks, but there are plenty of situations where you need to restart the repl: adding libraries, servers binding to ports, making sure you have no temporary edits, etc, etc.

True that temporary edits could in theory get in the way, although I never had that happen to me in ~6 years of experience with Clojure. Server bindings to port should not warrant a REPL restart, usually you can save the returned function/data from starting the server to also turn it off (and unbind the sockets).

> The whole 'reloaded workflow' is a workaround to the slow restart IMGO

I disagree with this, it's not a workaround. It's a different workflow, yes, but it is different on purpose, not by accident.

Being able to evaluate and run snippets of code will (for me) always be a faster workflow than even the fastest edit-compile-run languages, since I can evaluate code in the middle of functions and won't have to have any unit tests just to test something temporarily in isolation.

Thank you fogus and the team for your work on these tools!

Clojure aficionados love to obesess over this stuff or maybe it's a leftover from their Java days. You come away from Rich Hickey's sermons on the Mount all fired-up with the value of simplicity and before you know it you're knee deep in Clojure's build tools maze - lein, boot, deps.edn, clij, shadow-cljs, tools.deps, tools.build. Clojure blazed the trail for simplicity but as far as build tools are concerned only Go managed to pull it off.

Reply on news.ycombinator.com