State of Clojure 2016 – Results and Analysis
blog.cognitect.com
blog.cognitect.com
About 6 weeks ago, I fired up Emacs and tried out Clojure for a new project, and I have to say, it's definitely different and all of the changes are for the better.
1- Cider. I don't know how I lived without this years ago.
2- Security. Ring comes with CRSF tokens by default.
Friend was kind of a pain to use, and this meant that everyone was rolling their own back then. This isn't an issue anymore. That Bedra talk really gave the community some focus, and I'm glad everyone has been working on it.
3- I'm glad to see that libraries have been stable, improved upon, and not abandoned. While this includes lein, of course, so many things that aren't integral, like hiccup, overtone, etc, have been kept up to date for the new versions.
4- Better integration of libraries.
5- Much much much better build experience. Back then, run lein and have fun figuring out what you have to change in what file to get everything up and running. This is no longer the case, everything just works.
Of course, there are still some issues, but I just wanted to focus on the good stuff here.
Maybe I'm just used to the error messages, but I'm surprised "finding libraries" wasn't a higher-rated pain point. In many languages (at least in the web dev world) if you find a library that hasn't had commits in a few years, you can safely assume it's out of date. Not so with Clojure, and while this is due to two Good Things -- the stability of the language and the general avoidance of do-everything libs/frameworks -- it still makes it a bit hard to evaluate whether it's a good idea to rely on a given open source library. And good luck sorting through all the ring-<whatever> middleware to find the right one or right mix for you.
That said, I wouldn't go back to non-Clojure tools (for me that's Rails & JS) for most of my use cases (still awaiting a bit of maturation of the react-native wrappers for doing mobile apps. It's possible the benefits already outweigh the compilation/reload time cost, though).
I want to embrace a functional paradigm. I want to lean on Clojure because of the versatility of also being able to leverage ClojureScript.
So far I have hit some bumps that are a bit frustrating.
1. I changed too many files at once and the stack trace error I got gave me literally 0 clue on what was wrong. It was such a feeling of doom that it scared me on how to debug Clojure apps. The stack trace was full of Java errors and nothing pointing me in the direction of where the app was failing from Clojure's perspective.
2. Momentum of Web-Development and embracing new technology. I really believe in GraphQL and there isn't much support or conversations with using GraphQL with Clojure and ClojureScript.
3. I want to code Lisp better, but I'm not sure how to learn all the tricks on using Paredit. I use Spacemacs in Vim mode and a lot of the tutorials I see are of course in Emacs bindings so getting the know-how of navigating s-expressions in Spacemacs evil mode is quite the challenge but I'm going for it.
4. Boot up time is lame but it is what it is. `lein run` then `lein cljsbuild auto` can take ~20-30 seconds on a relatively fresh project on my MBA. After the initial boot time it's fine though.
Overall I have my head up high but those are some of the things I'm struggling with. I really want to love Clojure/lisp. I worry most about the community not being large enough to share examples of how to do new things. Which of course can be argued for the better (not a trendy community) but it instills worry nonetheless, especially when one really wants to adopt a tech all around (like GraphQL). I have an argument in my head that shouts "Well, if there's so much momentum in Javascript/Node for that ecosystem then might as well use that. Or even Elixir with Absinthe" – in the end it's about productivity right? :) Anyway rambling at this point.
For example, you should never change a bunch of code in different files and then try to run it and hope it works. Clojure workflow would be to change a particular function, send it to the REPL, see that it does what you want, then change the next function, and so on.
ClojureScript equivalents to GraphQL would be Om Next http://hueypetersen.com/posts/2016/02/13/om-next-from-a-rela... and and Posh with Reagent https://github.com/mpdairy/posh It's the same concept expressed in a Clojure idiomatic way.
I would highly recommend using Vim fireplace, Cursive, or Atom to start learning Clojure. Emacs is a very complex editor that takes a lot of time investment to be productive in. There's no reason to try and learn it while also learning Clojure. I've been developing Clojure professionally for the past 6 years and I've never used Emacs for that.
Second, I can't embrace Clojure specific idioms that remove other platforms. That's why GraphQL is amazing, it standardizes across platforms for many types of clients. I embrace a polyglot engineering structure and there will be teams that use and consume GraphQL on languages outside of Clojure.
I'm comfortable with Emacs (just in VIM mode ;)).
Edit: to add, momentum is important in my eyes as well. It's hard to put down the momentum of GraphQL and it's adoption for something niche specific unless the pros heavily outweigh the current momentum. Which is definitely possible (why I'm investing this much energy to discover the viability of Clojure/Script/idioms) but I get very skittish when I see the current issues of community size, errors, and the lack of momentum compared to other technologies (Clojure looks like it's popularity has somewhat peaked unfortunately).
If a company/team can leverage Clojure and it's niches as a secret weapon despite it's smaller momentum then that's a huge win. But that means it has to be that big of a win compared to alternative technologies that do the same thing with a larger amount of people familiar with it.
The thing with Emacs is that it's a programming environment in which you're supposed to program. People want it to work like they want out-of-the-box, but that's like asking the Ruby interpreter to write your RoR application for you. With Emacs you don't reach median productivity on day one but maybe after a week or so, but then on your productiviry will increase and increase. But with other tools maybe you reach the median productivity on day one or two, but you stay thereabouts thereafter. And they eat up all your ram.
I use Emacs (Spacemacs, so maybe that doesn't count!), though, and I'm willing to live with this if it means it can do what it does! :)
I only use the repl for exploratory-figure-this-out type work. Once I have it figured out, I rewrite the function and reload the environment.
I find it is too easy to have a messed up environment (state) when tooling around with the repl.
For a long time I got by with creating a split-screen terminal with 'screen', and then used vim-slime (https://technotales.wordpress.com/2007/10/03/like-slime-for-...). It doesn't have to be split, I just like it that way, but the idea is you have essentially two views talking to the same screen session. In one you launch a repl (with lein, or even a ruby/python repl if you want) and on the other half you open vim and type things like ctrl+c etc. to send fragments from vim to the repl.
More recently I've been using slimv (https://kovisoft.bitbucket.io/tutorial.html and https://github.com/kovisoft/slimv -- and if you want my three vimrc prefs http://pastebin.com/VjwrEJJQ) after using it for Lisp, which uses swank, and found I could also do some of this for Clojure's repl (https://github.com/technomancy/swank-clojure). Unfortunately it's kind of outdated, that's not where the community is going (and I haven't really bothered to figure out exactly where that is, though that project's readme gives hints, nor have I really tried Fireplace recently enough to say if I still dislike it). But it works for me, for now, unless I want to use ClojureScript. I'd say give it a shot, you might like it.
What you really want is just to run
lein repl
And it should all work automatically. With the new versions of vim-fireplace it does just work automatically.
I'll write a 'process-data function in Vim, send it with :Eval, and then test it in the repl with (process-data {:some "data", :more "other data"}) and it all just works. There's a little reading up on namespaces necessary once your projects get complicated, but it works on a bare repl right out of the box, too.
Does vim-fireplace easily let you replace forms with values (or macroexpands) or evaluate a form and have its value put somewhere? By default slimv lets you save the expression you're evaluating (which is useful for testing things) but I think I'd have to set something custom up if I wanted it to store the value somewhere or replace the just-evaluated expression. If vim-fireplace did that out of the box I'd probably consider it again. I know emacs can do it, watching people perform/livecode with Overtone et al. is instructive.
Other things I appreciate about slimv are when I type "(defn " I see a nice helper below saying "(defn [name doc-string? attr-map... prepost-map? body) + attr-map?])", this applies to everything. If I type ",s" on any symbol it'll show me the (doc) for that symbol, ",h" takes me to the online clojure docs. When I type tab it auto-suggests core functions, presumably through standard omnicomplete features though. (Edit: another useful thing, I can simply trace/untrace function calls with ",t".) Does vim-fireplace support all/any of that out of the box? It sounds like it supports my "screen" flow just fine these days but what else does it give me?
Are there any good HOWTO, tutorial or documents that explain how? (ahh "clojure repl" on youtube) what about text? Looks like I'll search. One hint as to the maturity of a language ecosystem, is official docs that show these kinds of things.
3. I would recommend you look into Parinfer - https://shaunlebron.github.io/parinfer/. You can manage scopes with levels of indentation, rather than having to move parens around. It works with most major editors, including Emacs.
But the Neovim port is great. Installation requires a little patience, but it works well.
In any case, one should read the Clojure style guide on GitHub [0] and get used to writing code in normal editing modes first. Then move on to Parinfer or Paredit once you feel the shape of the code and sense why it's like that.
If you've found a bug with the port, please report it here: https://github.com/oakmac/parinfer-viml/issues/new
Thanks :)
For the editor, I found a happiness with Atom + Proto REPL + Parinfer + Paredit (barely used). I wish I could get better integration with linting tools. I tried the vim approach for a while but Atom was way ahead in terms of REPL and Parinfer support. I'm not paying the cost to switch to emacs.
[0] https://github.com/WildUtah/secret-personal-dotfiles/blob/ma...
I just use the vim aspect of spacemacs for efficient navigation.
Regarding #4, yeah. It is what it is... lein makes it all about twice as slow. 30 seconds sounds pretty crazy, but I don't do a lot with clojurescript. Most of my stuff will startup in about 5 or so seconds with lein, 3 without. even on a wimpy vps.
It offers you all the cool features of paredit (barf, slurp, splice, transpose, convolute etc...), but it is easier to use in evil-mode. Check it out!
It is easily the best language/platform I have used, and it is a joy to work with, Leiningen and Figwheel especially. Being able to use this wonderful language through the entire stack is a bonus. The drawbacks listed are real, but are getting better all the time. Spec goes a long way to improving error messages.
That's not universally the case, but definitely a frustration. The examples are frequently the only way I can begin to figure out the usage of something at all (and they're typically the simplest-case example). I love examples in general, but it still speaks to the difficulty of parsing through some of those function documentations.
clojuredocs.org is just one community-powered docs site; there are several.
It also seems like there's some demand to fund an open source version of Datomic:
https://news.ycombinator.com/item?id=13583620
Crowdfund open source Datomic alternative here:
Note that accumulate-only is a semantic property, and is
not the same as append-only, which is a structural
property describing how data is written. Datomic is not an
append-only system, and does not have the performance
characteristics associated with append-only systems.
What I've been thinking about about is step in the datomic direction, built on top of plain old postgres that wouldn't attempt full time travel, but would use the same general schema/model with history. It's seems crazy that in 2017 that we don't have a decent general option for most business apps to store facts that doesn't over-write the past as is done in current sql databases.
It's not clear just how much of a demand there is for an open source alternative to Datomic which I'm trying to measure with the sign up landing page.
PostgreSQL solution might work if there was a way to guarantee immutable audit trail. This is really why Datomic favored in financial industries as it gives a high level of confidence that the audit trail can't be manipulated (even if you delete a record, that action will show up in the audit trail).
I do agree that it's rather surprising that we still don't have something like Datomic on github with a liberal licensing like MIT or BSD.
It could be due to demand or high cost barrier associated with emulating something like Datomic. And this is what I hope crowdfunding will change the game. Pay people to work on their pet project full time by crowdfunding it.
This actually works pretty well, but I have a REST API fronting the whole setup - programs are not directly accessing the underlying table or views directly.
Have overheard of this in passing. Might elaborating on setup/use-case? Seems strange to me for someone to restrict themselves artifically and not expose the whole power of SQL etc..
Direct access to our HIPAA/IRB database instances are based on white-lists - most internal and all external teams are prohibited from direct database connections. So for those groups, there is no possibility of any SQL access to schemas.
We use API keys with a shared secret (like AWS) to sign REST requests. If internal and external groups need to work together to jointly CRUD some resources for a specific project, they can share an API key dedicated to that project.
There was also a push to switch from an app-centric view of data management to an API-driven approach serving hypermedia (collection+json). That worked conceptually, but most API users ignore the embedded hypermedia links. I find that too bad, as those links make generic API consumption clients much more resilient to change - following a link embedded in a response rather than relying on POST to a well-known URI is a useful abstraction.
Oh and Spec is a godsend.
Which the clojurescript compiler devs have addressed in recently released alpha features including extern inference: https://clojurescript.org/guides/externs
Is there a well maintained and high quality repository of third party externs, as there is in the form of DefinitelyTyped for typescript type definitions? Would such a thing subsume https://cljsjs.github.io/ ?
Which the article mentions.
My team is moving in the opposite direction with some of our services. We currently run everything on the JVM with Clojure, but now we're experimenting running some services using Node.js due to its lower footprint.
As a Clojure fan, I find the lack of growth a little disturbing. Why isn't it 20,000?
Finally, I think the next version of Clojure will be able to make a big step. Spec improves multible problem points, error messages is one of them. It will take some time until the tooling around Spec is ready, but the data that spec provides is very nice for tools.
https://news.ycombinator.com/item?id=1118986
http://stackoverflow.com/questions/491376/why-doesnt-net-c-o...
Edit: typo
Does anyone have any suggestions or resources for how to keep the upfront memory cost of the jvm to a minimum (and monitor that its still within bounds over time).
128m is not acceptable, for my uses clojure would need to compete with go and rust which I can deliver 10-20m running in memory (and lightning fast).
I don't expect 10-20m but 10x doesn't work for me.
Think of deploying to Sandstorm.io as my goal ie small personal servers.
I wrote about the motivation for the project in some detail here http://yogthos.net/posts/2016-11-30-Macchiato.html
You can see some example projects here https://github.com/macchiato-framework/examples
I've already ported some of my JVM services to it, and I've been very happy with the results. There's a Clojurians slack channel #macchiato where project discussions are happening, and others have been using it successfully as well.
I don't see Node.js as a replacement for the JVM, but it is a better fit for certain types of applications. Being able to develop Clojure using essentially the same stack on both platforms means being able to choose the right tool for the job.
I'll probably pursue the jvm for a little while longer but I can see the attraction to clojurescript for this and will check in down the road.
I'm just kind of surprised and disconcerted how little I could find on the web, specifically for minimizing footprint.