Why Clojure? (2010)
thecleancoder.blogspot.com
thecleancoder.blogspot.com
"Here's a haystack where the error /might/ be, have fun finding the needle dipshit." Maybe I'm just spoiled with Elm, Rust, and Elixir's error messages, but the last time I tried Clojure (more than a year ago) I just hit a wall when I tried to make a toy app, as the Clojure compiler seems to hate me even more than C's compiler does.
Has this situation improved? If so, I'd love to take another stab at it.
See, http://clojure.org/about/spec
Its still not great, but improvments are happening.
Most of the time you're working from the REPL inside your editor anyways so the iteration loop is much, much better than any other language's. Even without clj-stacktrace I'd still prefer Clojure development to anything else.
Rebooting a program and losing state is always worse than error messages taking a bit longer to understand at first.
1) bad error message. There error is explained in terms of some other machine, but not on the language level.
2) a zillion lines of useless stacktrace information: I don't want to debug the REPL, but the wrong call. The stacktrace also lacks argument information.
Generally I would hope for a 'live' backtrace.
> Most of the time you're working from the REPL inside your editor anyways so the iteration loop is much, much better than any other language's.
Any other language, but not compared to those who have a similar or better REPL or debugger. Which is a zillion.
> Rebooting a program and losing state is always worse than error messages taking a bit longer to understand at first.
Not sure how that is related...
Not sure if it's _always_ worse. I've worked in imperative & functional languages, and the quality of error messages is key to solving bugs and understanding a sufficiently complex system, regardless of what language it's coded in.
You often have much more information at runtime than at compile time. Sure I can run the compiler and scan its errors but I feel its much more productive to poke around the program as its running. Even when working in C++/whatnot I learn more about programs by running them in the debugger than reading the code (reading UE4's source code will be weeks of head scratching because its so huge - 2 days of stepping into its boot process and you know and _understand_ most of its architecture).
I'd argue that the hardest bugs to find are the ones your type-system is completely helpless against. Few languages protect against null pointers and every time you have a cast you're breaking out of the type system and its guarantees. Dependent types aren't frequent either so you're now augmenting the type-system with asserts and guards and whatnot.
I find that in C++/Java/C# complex systems are the norm because everything is built out of mutable blocks and misconceptions about what makes programs fast.
Every project I did in Clojure was a fraction of the complexity it would've had in imperative languages. You're reducing complexity _so much_ that the "type-system is good for complex bugs" argument almost vanishes.
Agreed - in Ruby, most of my development was driven by a REPL & debugger, and tests. That's an example of a primarily imperative language, though it borrows some ideas from Lisp. But there, again, understanding error messages was key.
I see it more as a development library and don't ship it to production.
There's also cljs-devtools and dirac to enhance the chrome devtools with ClojureScript support. Once you try them you're not going back :)
Clojure is a hosted language, CL isn't. There are loads of tradeoffs from that design decision alone. Saying it feels primitive for that difference only is jumping to conclusions rather quickly! It feels like you're comparing decades of CL experience with days of Clojure experience :)
There is nothing in the CL standard which says a single word about it. The idea of the standard is that it is possible to have different implementations: native on some CISC/RISC/Lisp CPU, on top of C, on top of a virtual machine, on top of the JVM, using the LLWM, etc etc...
> That wouldn't be possible on the JVM. CL's condition system is unique to CL AFAIK.
Let's run ABCL on the JVM, on my ARM-based ODROID:
Armed Bear Common Lisp 1.3.3
Java 1.8.0-ea Oracle Corporation
Java HotSpot(TM) Server VM
Low-level initialization completed in 1.242 seconds.
Startup completed in 6.577 seconds.
Type ":help" for a list of available commands.
CL-USER(1): (handler-case (and (evenp 2)
(/ 3 0)
(oddp 1))
(error (c)
(princ c)
(values)))
Arithmetic error DIVISION-BY-ZERO signalled.
CL works on the JVM too (ABCL). There is a "java" layer where you can introspect classes and create them dynamically, etc. The syntax is not as terse as Clojure one's. There is one thing that I have had difficulties to do in Clojure, for example, is when you want to abstract over types:
(fn [type] (.. type (staticFunction)))
You have defprotocol to dispatch based on types in clojure. There's deftype to dynamically create classes and Java has reflection to begin with. You can call .getClass at runtime (which is what the type function does).
I did a little bit of googling for Clojure libraries that do this: https://github.com/zcaudate/ribol and https://github.com/clojureman/special
I don't actually recommend any of these and haven't tried them. In particular I think using a non-standard library for your error handling, which is already probably relatively poorly tested, is probably a bad idea.
I just wanted to point out that it's possible. I also think I remember something in a Rich Hickey talk (maybe "Clojure for Lisp Programmers"?) where someone in the audience asks about condition systems, and he responds that Clojure doesn't have one, but that it should be possible to build it.
One concern I have is how well they integrate with the native exception system of the VM and what stopping on an error waiting for programmer input would mean in a lot of contexts.
For example, stopping in a request handler while the programmer fixes the code will most likely trigger a timeout on the other end completely killing any advantage the condition system had in the first place.
You've effectively bloated your programs with libraries you're not even taking advantage of :)
The other advantage of the condition/restart system is that it makes your libraries more reusable: rather than having either to choose one error handling strategy or to construct a system for choosing an error handling strategy, the language itself provides a construct for making several error handling strategies available that higher-level code can choose between in a straightforward manner.
You rarely reboot a program under development in Clojure - you rather connect your editor to its network REPL and live code everything from there.
This makes not having a condition system a minor inconvenience at best.
However, this style of development has taught me to write code that's resilient to such errors in the first place (as it, errors wont leave the program in an invalid state), and the end result is that I feel much more confident putting code in production and restarting computations at the REPL is effectively free and feels the same as using the condition system for the most part.
Or you're developing your function in a sandbox and restarting the computation has no difference to resuming it in the first place.
Going away from falsely linear time and shared mutation is one strength. You can rely on what has been assumed much stronger. Very good for concurrency.
Links for anyone interested:
https://www.infoq.com/presentations/Value-Values (1 hour version)
https://www.youtube.com/watch?v=-6BsiVyC1kM (1/2 hour version)
I think it should basically be required reading for any programmer, it's a very easy to follow look into the functional paradigm. It's also free to read online! http://learnyouahaskell.com
FP => I'll have a Sazerac
Imperative => Excuse me sir, could you take some ice, add rye whiskey, add bitters, add absinthe, shake, strain into a glass, and add a lemon garnish before bringing it to me
Would you mind actually making Sazerac in your FP "analogy" as well?
(-> {}
ice
(rye :2-fingers)
bitters
absinthe
shake
strain
garnish)
All this shows is that OO and FP are duals [1]. I don't claim to get FP perfectly, but my moment of zen was realizing this.
The value for FP comes from proper abstraction over these processes in functional terms, at which point they can be trivially implemented (few bugs, few iteration cycles to get right). This can probably be done for every problem space, question is at which the abstraction costs outweight the gain. Considering that FP becomes more and more mainstream it's probably more viable than thought in the past, still I imagine system-driven games or complex real-world simulations, with lots of side effects, would lose more from FP than they'd gain.
Some time ago I thought that, too, but I'm no longer convinced. Directly mutating values (OO style) feels more natural at first, but then you have trouble with side effects and order of execution matters more than it should, you start keeping snapshots of the whole state just to get a consistent world state during computation, otherwise this whole mess produces a whole class of bugs on its own. These problem drive you more and more into FP direction, and the FP style definitely does have its merits in this regard.
I think the following article articulates this very well:
"A Worst Case for Functional Programming?"
Ultimately it's the same result? The difference is when you can reuse and compose functions.
FP you start with the methods and just keep composing. With OO you start with classes and objects.
Me(Drinker)->drink( Sazerac(Cocktail) ->garnish() ->strain() ->shake() ->absinthe() ->bitters() ->whiskey() ->ice() ->glass() )
(def sazerac
(-> (mix :ice :whiskey :bitters :absinthe)
shake
strain
garnish))
(drink :me sazerac)
sazerac = do
add ice
add ryeWhisky
add bitters
add absinthe
shake
strainInto glass
add lemonGarnish
main = serve $ makeCocktail sazeracEither you have an object with all of garnish(), strain() and whatnot on it, or each method returns the object to handle the next step in the chain. Either methods don't scale at all without modifying existing code.
The real difference is that function composition gives you a reusable function you can further compose while objects keep piling up methods until you're left with god objects or indirection hell.
Reminds me of the blub paradox in beating the averages[1].
Not necessarily; you can easily write instance methods which merely copy the existing object.
In FP you will end with garnishFingerFood and garnishCocktail because you need to encode somewhere a specifics of garnish action. In OOP you will have garnish methods on Coctail and FingerFood and specifics and related knowledge how you need to perform garnish will be on object itself.
OOP is really powerful concept but failing in languages that have shit implementation. Java, C++ forsake OOP principles for "performance" or are made by people that do not understand concepts (Python, PHP).
The comment that FP isn't nice in the real world is pure baloney. For lots of "real world" IO-bound types of problems there is nothing better suited than a functional programming language with powerful abstractions. Things like monads let you write code in an imperative style without losing any of the benefits of writing in the functional paradigm.
You're complecting. In the real world of Clojure, you could simply define a protocol and provide different implementations of "garnish".
(defgeneric garnish (what with-what))
(defmethod garnish ((c cocktail) (f fruit)) ...)
(defmethod garnish ((s sandwich) (h ham)) ...)
...
(let ((currified (rcurry #'garnish :curry)))
(map 'list currified items))These look very much like Clojure's protocols and multi-methods.
Most of the time however I try to avoid aspects as they introduce hidden behaviour in existing functions, which can be hard to reason about at scale, especially with more than a few developers.
Aspects I feel are more useful when you want to modify the behaviour of existing code you don't own.
Maybe if I had easy access to them I'd find more use cases :)
The imperative equivalent would be:
"Give me a Sazerac!"
(imperative, hehe)
Second, that's not functional programming, that's calling a library function.
FP => I'll have a Sazerac
Imperative => Serve me a Sazerac
Imperative programming isn't devoid of abstractions; it just has different ones.
This is only a feeling. Without carefully testing the code to exercise its cases, all you know is that they are properly typed.
For instance, suppose we write a complicated function (or group of functions) which goes through a block of intermediate code (output of a compiler) and assigns registers to all the temporaries, introducing memory spills in situations where more variables are live than the available registers.
We might have a feeling that because this code compiles, it must be free of problems such as accidentally assigning the same register to two variables which have overlapping lifetimes.
That feeling is poorly supported by reality; it is the "safyness" of static typing.
Untested code is garbage. And thorough testing is difficult to impossible, so there is a bit of garbage in almost all software, unfortunately.
Statically type checked code has all of its code paths effectively tested by the compiler, but those tests have only the limited point of view of trying to show that the program contains a trivial type mismatch, a close cousin of the syntax error. These "tests" are not actually feeding values into the code and trying to make it fail or behave incorrectly with respect to its specification.
Static typing is extraordinarily useful in large company settings, where you may own a library that is used across the company, and you can quite easily understand how your library is used in other codebases and can confidently make cross-cutting emergent patches.
I'm pretty optimistic in the direction that Clojure is heading with clojure.spec. The ideal that I hope it reaches is "inductive" type safety, where you can prove your program is typesafe with a certain probability. And the tradeoffs you get by relaxing the deductive properties and timeliness of static typing are vastly simpler expressiveness, composability, testing, etc.
The type checker just ensures that a program is well-typed (i.e. free of type errors).
The better the type system the more program properties can be encoded in it and the more errors it can catch (up to the point of proving correctness of your program).
But you are right that type checking alone is no substitute for testing. They are orthogonal concepts and both should be employed to ensure that your programs behave correctly.
One nice thing of statically typed languages is that they can automatically generate some tests for you, like e.g. the QuickCheck library for Erlang and Haskell.
That means that false positives are the problem for typing systems, while false negatives are the problem for tests. But that's the biggest difference you will find.
One does even replace the other.
Is there any studies done showing that strong typing is actually a disadvantage? I've seen some stating the opposite, but nothing really stood out to me. Sorry for the long rant, but at this point I can't find anything empirically stating one is better than the other.
Interesting experience recently: I added static checks to a Lisp dialect which report warnings for all occurrences of unbound variables and functions. (Not type checking, but a very basic static check). According to static proponents, bugs of this type should exist left and right if we don't have the check.
Only one instance of an unbound variable was found in the dialect's standard library: and that was a function that was added as an afterthought and never tested. The problem reproduced 100% when calling the function in any correct manner whatsoever; the function was completely broken. If it had been called just once after having been written, the problem would have been caught.
I fixed the problem so that the warning went away and caught myself almost starting to think about commit the fix, when I realized: what am I doing? I still haven't called the function. Gee, it now passes the one feeble static check that was added, so it must be correct? Haha.
Just a small point: type declarations are part of the actual logic of the program. In type theory, types are logical propositions and values are proofs of those propositions.
Has that become a thing now? :))
[1]: http://clojure.org/about/spec [1]: https://github.com/clojure/core.typed
The REPL-driven development of Lisps makes types not as important because everything is interactively tested as you write it. Having an optional type system makes it much easier to start with dynamic types and gradually add type annotations as your architecture stabilizes.
Dynamic typing does not imply lack of mutability. Those are unrelated concepts.
Python and Ruby are dynamically typed but both are certainly mutable.
Conversely static typing actually makes it easier to reason (locally or not) about things, since you always know the type of everything.
I don't feel static typing makes it easier to reason about, at all. Reasoning about values and their transformations is more important to me than reasoning about types, and since values carry types you're actually working with more information. Most of the time the types I'm interested in are closer to concepts (sequences, mappings) than concrete classes.
Thats where having immutable dynamically typed values is better than having statically typed mutable ones.
Only for the specific value that you are reasoning about. If you want to reason about all possible values, then you are de-facto reasoning about their types.
> Most of the time the types I'm interested in are closer to concepts (sequences, mappings) than concrete classes.
But sequences and mapping are also types, right? You can abstract types to type classes (i.e. whole classes of types, e.g. all types that can be iterated over, or all types that are ordered etc.) and also reason about them.
> Thats where having immutable dynamically typed values is better than having statically typed mutable ones.
Sure, ideally you want to have immutable statically typed functions and values, since you can then do some equational reasoning and prove certain properties of your program.
True, but a variable can only ever have one value at any given time; knowing a variable is an int is good, knowing a variable is the same int value throughout its extent is better :)
> But sequences and mapping are also types, right?
Yes, but they're not concrete types and don't map to a single interface either. When they do these interfaces are compositions of smaller interfaces anyways and I prefer to reason about these smaller parts.
Which means I'm reasoning more about the shape of the data than the actual type implementing it, and that to me is closer to a value than a type. Its the same with type-classes, you're reasoning about the features of a value rather than the specific type implementing said features.
> ideally you want to have immutable statically typed functions and values
For functions we're talking about purity rather than immutability (which qualifies variables). Type systems also are very leaky abstractions; null pointer exceptions, can't limit the range of value, requiring more code which introduces its own bugs and complexity and more.
From experience I prefer a smaller dynamically typed codebase that has tests for the trickier parts over a statically typed codebase (even if tested). The productivity difference is startling and the resulting quality is about the same.
The shape of the data is its type. You have been reasoning about types all this time. Check out https://en.wikipedia.org/wiki/Structural_type_system
In a language like Ruby or Python, it's possible for a side effect to change the type of the variable you're looking at externally. This cannot happen when you're writing idiomatic Clojure.
Static typing comes at a cost however. You have to figure out how to encode the problem using types. Effectively, you have to prove to the compiler that your code is self-consistent. Proving something is often much harder than stating it.
At the same time, static typing doesn't guarantee semantic correctness. So, it doesn't actually tell you that your code is correct in any meaningful sense.
I think that Clojure Spec packaged with the 1.9 release, provides a much better way to catch errors than types. The main reason being that it focuses on ensuring semantic correctness.
For example, consider a sort function. The types can tell me that I passed in a collection of a particular type and I got a collection of the same type back. However, what I really want to know is that the collection contains the same elements, and that they're in order. This is difficult to express using most type systems out there. However, with Spec I can just write:
(s/def ::sortable (s/coll-of number?))
(s/def ::sorted #(or (empty? %) (apply <= %)))
(s/fdef mysort
:args (s/cat :s ::sortable)
:ret ::sorted
:fn (fn [{:keys [args ret]}]
(and (= (count ret)
(-> args :s count))
(empty?
(difference
(-> args :s set)
(set ret))))))
Not really. When you learn the properties of your type system, you can 'prove' your way through most day-to-day programming tasks fairly easily. And when you can't prove something to the compiler, you can just fall back to telling it using its 'dynamic' escape hatch. You're certainly no worse off than you are with pure dynamic typing.
> ... Clojure Spec ... focuses on ensuring semantic correctness.
Nothing in a static type system precludes something like Clojure Spec, or at least something very much like it. You can still fall back to runtime tests that your function works according to spec. In fact that's what Haskell's QuickCheck and related generative testing systems are all about.
I am not surprised at all Bob Martin loves it. Any principled software engineer would.
[1] https://skillsmatter.com/skillscasts/2302-radical-simplicity
The feeling you get with Clojure is that, when you write and test each function (using CIDER) and get live feedback to make sure they work, you can combine them into bigger functions that most likely also run correctly.
I'd much rather have SBCL's native code compiler, read/compiler macros, conditions and restarts (that I end up using on pretty much every project) and optionally use libraries for immutability and STM (if and when I need them), than compromise from the get-go and use a language that reduces the set of available options by forcing its specific worldview.
If I do need a strong focus on concurrency, I find Erlang (and also Elixir) a much more coherent solution. The cognitive dissonance that comes from having to interact with Java when using Clojure is very damaging and it can't be abstracted away. Just look at Clojure stack traces.
So to end with, Clojure proponents should understand its limitations and design choices. It was created because Hickey needed a language to solve problems he was having in his specific arena (delivering concurrent applications in the Java ecosystem with his consultancy) and not really to improve on the Lisp state-of-the-art side of things. If you're a Java guy and absolutely need to stay in that ecosystem, I guess you can go with it. Otherwise, you end up giving away too many things. This has been validated in practice, in the Common Lisp community. We get a lot of guys who are coming _from_ Clojure, but it is rare for someone to move _to_ Clojure from CL.
It's true that Clojure isn't trying to be SBCL, but you're mischaracterising what Clojure is designed to do.
Clojure is based around the idea that good code minimises interconnections. In Clojure parlance, "complex" code is code that is very interconnected, and "simple" code has few interconnections. Clojure is designed to make it easy to write "simple" code.
CL programmers tend to look at Clojure and say "Why should I use Clojure when CL is more powerful?", but Clojure isn't trying to be more powerful than SBCL, because sometimes adding power means sacrificing simplicity.
Clojure is an insufficient CL, but CL is an insufficient Clojure.
This is what has kept me from really feeling comfortable getting into Clojure. Unfortunately there are so many downsides in the Functional programming world :( It's either:
1. Have a strange interrop (Clojure with Java, and even though Elixir with Erlang isn't as bad, it still bothers me)
2. Lack of mainstream adoption (e.g. Haskell, yes FB uses it for their spam filters.. but mostly working developers predominantly use other languages)
3. Lack of clear tooling choices or implementations (Haskell's stack vs. platform, or Scheme is great but no one popular implementation)
(I know I'm being quite pessimistic)
I don't use Clojure, but isn't relatively seamless Java interop -- and thus access to the massive Java ecosystem -- considered one of the big pragmatic advantages?
> Just look at Clojure stack traces
There are prettifiers that clean this up, but I understand your complaint is cognitive dissonance. The Java guts are right there, and I guess this is the heart of that matter. Some don't like chocolate in their peanut butter.
Developing in Common Lisp is perfectly doable by editing then saving a file and reloading it in, for example, SBCL.
I think Agents are more on the way out because of core.async.
A great language that does optimize mutually recursive tail-calls, among many other elegant features: Lua!
... TCO, asymmetric stackful coroutines (90% as powerful as call/cc, but with zero calories), lexical closures, prototype-based object orientation, duck typing-based object orientation (among other possibilities), and a first-class C API that allows C code to work cleanly with coroutines, closures, the object system(s)....
Lua is truly multi-paradigm. The only downside is dynamic typing, though that's not always a liability and often an asset. Also, Lua has other interesting features, like lexical global namespace substitution using _ENV, that permit devising solutions to minimize some of the headaches of dynamic typing. And because Lua is so powerful in so many dimensions, yet so simple and tiny, writing unit and regression tests is often a breeze.
Once you add the canonical extension module LPeg into the mix (written by one of the Lua co-maintainers), Lua is about as formidable as they come. Only Perl 6 comes close to the ease of writing parsers with Lua+LPeg.
[1] The JVM will eventually add support for implementing TCO in Clojure and other languages, at which point many Clojure proponents will find religion in the power of TCO.
For mutually recursive TCO there's `trampoline`
What I was getting at is: in an automatic TCO language you don't actually know if the call is optimised or not. You might think it's optimised, but it isn't, because it's not in the tail position. You don't find this out until one day in production the stack explodes. The only way to know if it's optimised is to determine if it is in the tail position. That sometimes is straight forwards, but sometimes requires careful thought. It is certainly not always obvious.
Additionally, another coder can come along and break your tail position. You've gone "return func()" and it's TCO and then later someone changes it to "return 1 + func()" and now the call to func is not TCO, but there is no warning and no obvious outward signs that this has changed.
In the explicit TCO of clojure, if the second programmer adds the extension the compiler will explode with something like "recur not in tail position", immediately informing you that this has happened, and then you may either refactor to put it back in the tail position, or change it to a full stack function call once you determine that building up stack is ok in this case.
Now I think loop/recur was implemented in Clojure as a work around the JVM limitations, but in doing so I think Hickey stumbled onto a really great new syntactic formulation for TCO. Explicitly declared TCO. I like it.
(define (sum xs)
(let loop ((acc 0)
(xs xs))
(if (null? xs)
acc
(loop (+ acc (car xs)) (cdr xs)))))
The named let construct is more general than loop/recur, since you can supply a name and properly nest the forms.
It will TCO automatically. If you want to make sure you haven't made a mistake, simply add the @tailrec annotation, and the compiler will give you an error if it can't TCO.
If you think about it, you can't really do the opposite, because it is not possible to TCO every recursive calls.
I no longer believe in transpiration or hosted languages. Proper language have LLVM backend. Look at Rust, Swift, Crystal, this is future.
Clojure is more Java based than JVM based. This is main issue. It inherits all Java limitations and share problems inherited from Lisp world. https://github.com/clojure/clojure/blob/master/src/jvm/cloju...
By selling soul I mean: 1. Clojure will forever have ugly/broken stacktraces. 2. Inline/Class limits on JVM will not disappear. 3. Half of Clojure libraries are just wrappers. 4. Lack of TCO and other recursions optimisation. 5. Performance will be always worse than Java 6. Type system with garbage in -> garbage out approach
JVM execute bytecode in Java Virtual Machine. JVM abstract your hardware and have specific runtime that needs to be present.
LLVM is superior model because its avoid additional compilation and can in theory provide much better performance.
As for the runtime being present, it's more of a packaging issue. A language like Go has a runtime that's similar to Java's (except it doesn't JIT-compile), but it is statically linked with your binary. This can be done in Java, too.
I must say that it doesn't seem like you're very knowledgeable when it comes to compilation techniques. LLVM and the JVM do make different tradeoffs, but they're not at all the ones you describe.
Only better if you don't include the runtime compiling (start-up time).
Look, I like tail calls as much as any language geek, but this statement is just plain silly. The idea that TCO is needed to "fully" leverage anything is silly. Never mind that necessitating so-called full leverage is pointless, as even supposedly partially leveraged anything can be useful. In the case of Clojure, function composition and immutable data structures are highly leveraged and, after learning the seq abstraction and lib functions, you won't even miss tail calls unless you try to write a direct-style interpreter for a language that supports recursion.
Functional programming relies completely on recursive functions. You just can't do "full" FP without tail call optimizations.
Now, I do side with the other poster up there that is talking about jumps. You don't even need jumps actually, you can use inlining and loops too, but they do make the compiler's life easier.
Yet that's such a small part of what FP has to offer that it isn't funny.
Extremist views are not particularly useful in engineering, despite their utility is science.
> The idea that TCO is needed to "fully" leverage anything is silly.
I provided two examples, monads and CPS transforms, which require TCO to fully leverage in a strict language. But really any example where functions are dyamically composed as a pipeline needs TCO to avoid leaking stack. Your examples of a couple of test libraries in Clojure hardly constitute evidence that TCO is of limited utility.
Sure, but in the rare case that this is a problem, you can trivially workaround it:
(reduce (fn [acc f] (f acc)) init [f1 f2 f3])
(reduce (fn [acc f] (println "executing" f) (f acc)) init [#'f1 #'f2 #'f3])
The Haskell research community is in the process of undoing this brain damage on the monad front by embracing extensible effects, free(-er) monads, "reflection without remorse", effect interpreters, etc.
Once you need any sort of symbolic reasoning, you drop opaque composition and work with data. This is true in every language, FP or not. People tend to do exactly the right thing already: abandon function combinators and write interpreters over data structures.
Regarding your comments on replacing function composition with data structures. In essence, this is what trampolining does. But it does have a performance cost and is not appropriate in all cases. IIRC correctly the performance of "Extensible Effects" is still an outstanding issue. Incidently, the primary author of that work, Oleg K, has published a lot of work in which he seeks to avoid intermediate data with "finally tagless".
I agree with the parent that lack of TCO in Clojure is probably not an issue for most.
In other words, Lua has nothing to do with LISP, except that it's your favorite, hence I don't understand this comment.
As for TCO, it's regrettable that the JVM runtime doesn't provide support, however you can work around it by implementing trampolines, which is how developers of Scala, Clojure and PureScript have managed to live. For example you can implement a fairly efficient and lazy IO type, like the one in Haskell, which then gives you memory-safe tail-calls for free. Examples include the Free monad or the Task type from Scala.
The only downside of this approach is that it involves effectively building your own call-stack and keeping it in heap, which stresses the garbage collector and implies some indirection. But then again, if you'll look at the TCO performance of platforms supporting it (e.g. .NET, ES6), TCO calls have a performance penalty which can be quite significant, so I'm not sure that you'd gain much in terms of performance.
And if you measure the performance of FP-heavy Scala and Clojure, you'll see that it's not bad compared with Scheme implementations or Haskell, in spite of trampolines being used for tail-calls.
No, Clojure doesn't have TCO because of a design decision. We can discuss the rationale behind that decision and the merits of it, but claiming it has anything to do with the JVM is dishonest.
Guess what the JVM doesn't have. Remember also that the JVM was originally designed for running untrusted bytecode, so it doesn't generally let code directly manipulate the call stack.
https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-6.ht...
> so it doesn't generally let code directly manipulate the call stack.
You are doing something wrong if you need to manipulate the call stack to do a tail call. You just have to update the local arguments and jump back to the start of the function. It is literally just turning the call into a loop.
TCO works for all tail calls, not just the recursive ones, so that jump would be to another function's body. The current stack frame may not have enough space for the new function's.
As long as you have control over all the code generated you can in principle compile multiple functions into a single one and just add some stubs for interop purposes. This of course complicates how you track function pointers (or equivalently) and may have some overhead.
Still the cases where you can use recur today could be done automatically by the compiler without much trouble.
One worry I have about automatic TCO where recur/loop currently is used is with Lisp's dynamic nature. What happens to everyone who has a reference to the fn when you redefine its def?
You're basically switching (defn f [] (recur [] (loop))) to (defn f [] (f)). What if I do (def g f) and then (def f +)? Will the original f now recur into + when I call (g)? If it still calls into the original f then that creates a mismatch between reloading normal and recur'd functions adding complexity to a language striving for simplicity.
I'm probably overthinking it :)
However Clojure binds at the definition time, e.g.
> (defn f [] 42)
#'sandbox9516/f
> (f)
42
> (def g f)
#'sandbox9516/g
> (g)
42
> (def f 1337)
#'sandbox9516/f
> (g)
42
> (f)
java.lang.ClassCastException: java.lang.Long cannot be cast to clojure.lang.IFn > (defn f [] (f))
#'sandbox9672/f
> (def g f)
#'sandbox9672/g
> (def f 42)
#'sandbox9672/f
> (g)
java.lang.ClassCastException: java.lang.Long cannot be cast to clojure.lang.IFnWhy? The only issue I can see is that you can't tail call to java code, but that would only be a problem if you are trying to do mutual recursion involving both Clojure and Java code.
For multiple Clojure functions with mutual recursion you could just compile all of them into a single function with a state argument to select which function to go to, and then add stub methods that just calls into the combined function with the correct value for the state argument for interop with java. The only artifact of that would be an extra method with a synthetic name in the stack trace.
EDIT: Also what you suggest would imply huge changes to the way code is evaluated in Clojure - for example, I can create functions in the REPL and they work the same way as reading functions out of a file. How would that work in your scenario? I think every function would actually have to have a stub not just the mutually recursive ones because you might not know until later which functions to combine.
So none of them are worthy of being FP languages according to you.
I would argue that it doesn't get any easier than using Rebol/Red parse dialect - http://blog.hostilefork.com/why-rebol-red-parse-cool/
Minilanguages... for everybody!
http://ecraven.github.io/r7rs-benchmarks/benchmark.html
(Chez is still freakishly fast.)
However, if you're writing code in DrRacket, it has full debugging information enabled, which may slow the code down. So your REPL performance in DrRacket may be slower than if you ran Racket from the command line.
Fortunately, DrRacket's friendly GUI makes it pretty easy to toggle debugging. I leave it in place most of the time, but occasionally the slowdown is just unbearable.
In particular, I've spoken with people who have cited significant differences in performance, when using proprietary compilers matched to vendor CPU chipsets, for Linux/OpenJDK compilation, which then changes the profiling of the JVM benchmark, on the same bare metal crate.
Meanwhile, L2 cache misses, due to object reference churn, are the final deal breaker with the JVM, if I've read correctly and read enough about JVM performance. Beyond even garbage collection overhead, fundamental JVM performance cannot be tuned to optimize CPU cache use.
Briefly paraphrased, the JVM performs messaging/referencing with String references, that look like "java.lang.Object@%HASH_CODE%" and when you instantiate and garbage collect many instances (millions of those hellishly long fully qualified java names) across many operations, costing many cycles, you'll always suffer incurred latency, because most of your objects are frequently populated and then evicted from the L2 cache, no matter how long they actually live in the JVM heap.
I don't have the link handy, or I'd post it.
But long story short, even though a JVM benchmark may be performant, benchmarks do not peg CPU resources according to real life use cases.
And changing the OS, motherboard, and compiling from source on your own physical hardware can provide benefits that might not be obviously worth the extra effort to compile from source bootstrapping all the way from the kernel to the JVM, before deploying your java artifacts.
Proposal: Fast for server use-cases?
Curiously, I was never really interested in the startup time of most of my code, probably because most of my code is long running where it doesn't matter if it takes a few seconds more to start.
Planck, which runs on JavaScriptCore: http://planck-repl.org/
Lumo, which runs on V8: https://github.com/anmonteiro/lumo
Last time I tried (which was probably over 2 years ago by now) the CLJ android had a huge loading time to the point of not being usable for anything serious (it offered no tangible benefits that would justify the load time it had) and iOS was something based on that JVM for Android port that Xamarin bought then shut down. Has that changed ? Or are you talking about CLJS + ReactNative and similar tech ?
Turns out, it's easy with Clojure. Have even written desktop apps with it since.
kotlin 1.1 (still in milestone) is a brilliant and compelling language to use on the JVM.
Spring 5 is a very "functional" web framework that will come out in a few months with first class kotlin support (even today, it is fairly excellent [1])
Vert.x [2] has incredible support for Kotlin and is coming up with built in kotlin support. Coroutines got merged fairly recently [3]
Reactor (https://projectreactor.io/) and Rxjava have had kotlin support for a long time.
The tooling is excellent (umm.. it was developed by Jetbrains).
The killer app/functionality ? Android. Kotlin is the swift of Android and that's where its uptake is coming from.
[1] https://github.com/sdeleuze/spring-boot-kotlin-demo/tree/all...
[2] http://vertx.io/whos_using/
[3] https://blog.jetbrains.com/kotlin/2016/07/first-glimpse-of-k....
I recommend updating the title to "Why Clojure is better than C, Python, Ruby, and Java"
The current one is much better.
At this point I would actually use Clojure as a filter. I'm rather suspicious of any developer who wouldn't be able to learn Clojure.
It's hard to find people (a) willing to make the jump and (b) making the jump effectively. And when you do, you can maybe tap into x number of devs for your team, but trying grow to 2x devs is much harder.
I also found that hiring students for junior positions from university works great as well. They don't have preconceptions about how to do development, and usually have easier time picking up new paradigms than devs who've been working in a particular one for a long time.
>but they were able to become productive doing useful work within weeks.
Sure anyone can learn clojure syntax in a day and write imperative code in it. But not sure if everyone can throw out their thought patterns from decades of OO and learn an entirely new paradigm of thinking about code.
One of the issue we had was precisely this, people who were just starting to write clojure were writing code that looked and felt totally different from people who have been doing it for years. Imagine paying this huge cost for every new dev in a large org.
>Sure anyone can learn clojure syntax in a day and write imperative code in it. But not sure if everyone can throw out their thought patterns from decades of OO and learn an entirely new paradigm of thinking about code.
We don't have to hire everyone though. We just need to find one person who fits the team. A lot of developers nowadays are already familiar with functional patterns, so it's actually not that challenging to start doing stuff in Clojure.
We also do code reviews and pair up as needed. Developers are a long term investment, so spending a bit of extra time to get somebody onboarded is well worth the effort in my opinion.
We're using jira/bitbucket internally. Our process is that everybody works on branches, and when you finish a feature you tag somebody for a code review. We found that this helped us catch any non-idiomatic code fairly quickly.
So far I've only ever seen terrible CTOs and directors rely on TIOBE to the point I can't take it seriously whatsoever. Choosing a language based on TIOBE will yield you an endless supply of mediocre resumes for your job applications most of the time.
I feel like a company using a language they've carefully chosen for the problems they want solved with a strong training program for new hires will have a huge advantage over a company solving the same problems using languages naively taken from TIOBE. From experience, the later company will easily spend two to three times as much in development costs and time for a product of lesser quality.
If you go with an esoteric language and then find new hires who already know the language, you know they weren't told to learn it. They were passionate enough to learn it on their own and these hires usually are golden.
I was using TIOBE as an agrument against a language for a huge company not as an argument for a programming language.
If you're already at 15k employees then pilot projects are the way to go; gradually incorporate the new tech in the company using low-risk projects. Their post-mortems will provide loads of material to decide whether its worth it as well as how to plan for formations should you stick with the tech. I've done it before, although not at 15k scale, but the method should scale all the same.
If you're a startup, going with TIOBE will probably help your competitors more than anything else really.
All languages were niche at some point. OS X was built on Objective-C and Windows on C++. I'd argue they both were niche languages at the time. Google spawned Go. Facebook spawned Flow. Microsoft also spawned TypeScript.
Programmer time is an extremely underrated metric and these niche languages are all attempts at improving it. If they succeed they stop being niche languages, at which point anyone already using them have a huge head start.
Companies not focusing on improving it are doomed to be replaced by those who do. Just like companies who kept to assembly are now history.
Apple was weeks away from bankruptcy when it adopted OS X, and is now one of the most successful companies in the world, having rescaled from the ground up on OS X. The iPhone's iOS is perhaps the most financially successful bit of consumer software in history.
However, you don't see big companies emerging that often anymore nowadays, so that could be a factor as well. And you don't see many startups relying on TIOBE either. Every place where programmer productivity is critical to the company's success is where you'll see niche languages used the most.
If startups would stop getting acquired as soon as they get profitable maybe we'd see big companies emerging while using niche languages.
- Clojure is a Lisp. Lisps are very elegant in their simple tree syntax, so compilers like them, and reasoning about Lisp code is a joy. However no one has yet figured out how to represent Lisp code in good a way that doesn't require a large number of parentheses, often stacked together in groups of three or four. If you have a slight astigmatism this (((()) will not help.
- It's a jvm language. That has gives all the benefits you want in a functional language such as type erasure meaning you can't make a data structure of primitive types without either pretending they are objects (boxing) or writing a custom type. Also, you have access to the vast amount of jvm libraries, almost all of which use mutable data structures and nulls everywhere. Jvm lacking proper tail calls will also give you the joy of having to manually specify when you want tail recursion rather than just recursion...from the tail.
FunCall1( FunCall2( FunCall3(arg1, arg2, arg3)))
(FunCall1 (FunCall2 (FunCall3 arg1 arg2 arg3)))
And the end of the line with all the parentheses. So much uglier to see three parens at the end of the lisp line than the non-lispy line.
Truly a monstrous imposition, that lisp syntax.
And when you look at any lisp code base, look at how they don't indent their code at all. So unlike other languages. Here's literally the first project I could find looking for "lisp source code": https://github.com/adamtornhill/LispForTheWeb/blob/master/we...
(-> (FunCall3 arg1 arg2 arg3)
Funcall2
Funcall1)
Well color me flabbergasted.
Fun3().Fun2().Fun1()
thing3.thing2.thing1
Concrete example from your linke: a projection and sort (I believe it is). I think that the tail of this expression with 5 parentheses makes this very hard to read (Admittedly it's a matter of experience of course):
(docs (iter (db.sort *game-collection* :all
:field "votes"
:asc nil)))))
Here is some half-functional half-imperative strawman syntax: for comparison:
x = some_collection
.select(c -> c.documents)
.sort(d -> d["votes"], direction: ascending);
> how many args does "iter" have?
Forget about parentheses and look at indentation. You need properly indented code, but that comes easily with a good editor:
(foo (bar ...
...
...)
...
(baz (+ x y)))
Likewise, "iter" has only one argument, because there is only one subtree at the indentation level where its arguments are expected to appear. If I try to add arguments inside the set of closing parentheses, emacs actually color them in red to signal that this is not easily readable.
http://dept-info.labri.u-bordeaux.fr/~idurand/enseignement/P...
It's obvious to me in a millisecond glance that docs and iter are called with one argument.
For one thing, the snippet does not contain a single instance of a ) parenthesis being followed by a ( parenthesis. Also, none of the ) parentheses have any material between them. It's just (sym (sym (sym stuff ...))).
Only three trailing parens are required to close (docs; the other two are for something else.
If iter had additional arguments, it might be written like this:
(docs (iter (db.sort *game-collection* :all
:field "votes"
:asc nil)
(another-iter-arg)))
That said: if the code is neither correctly indented or has the parens correctly balanced - then one is in a pickle. It's easy to balance parens on correctly indented code, and it's trivial to indent correctly balanced code.
But you can format the code automatically, and check if all is good by looking at the shape of your code (indentation). If the amount is the same but you put elements in a strange location, like:
(defun bar ()
(let ((a (foo)) (print (list a)))))
(DEFUN BAR ()
(LET ((A (FOO)) (PRINT (LIST A)))
))
Indentation and parenthesis introduce a level of error checking by redundancy: the tools works on structured expressions, you look at indentation. Here above you can see that the body of the LET is empty, while there is a bogus binding in it.
The above example is in practice quite rare. Most errors are easily spotted with paren highlighting. I use paredit, which makes structural editing easy (but which is not too rigid and allows me to make unstructured changes too). You generally don't have unbalanced parenthesis, and if you don't spot the bad syntax with indentation, the compiler has a chance to complain, too.
Hope you like Null Pointer Exceptions. (And poorly named methods.)
I prefer type systems without null if given the choice, but for composition there isn't much different between result|error and result|null (If the null/error case is actually handled. Unless you use C you can usually handle it with something that resembles modads or is at least a pattern matching shorthand
C#
var managerStreet = employee.Manager?.Address?.Street;
foo()?.bar()?.baz()?When I work with Clojure, I'm able to think in terms of blocks of logic as opposed to lines of code. This is extremely powerful.
The JVM is certainly not a limitation for many applications out there. Conversely, since Clojure has recur, the only time lack of TCO comes into play is when you're trying to do mutual recursion. I haven't run into a single case where I needed that in 6 years working with the language professionally.
However, Clojure is not really tied to the JVM. It runs on both Node.js and CLR. I'm actually currently working on a Node based micro-framework for Clojure: https://github.com/macchiato-framework/macchiato-core
Ah, not so! In some ancient Lisp dialects decades ago, there existed an invention known as the "super bracket". Namely the closing square bracket ] would simultaneously close any number of outstanding open parentheses up to the nearest opening [. For instance [foo (bar (1 2 3] could be written instead of (foo (bar (1 2 3))). It's not really such a great idea so it didn't catch on.
I think this is a pretty common syntax problem, it's for example why I prefer xml to json for human editable data despite the extra verbosity - each ending is explicitly labeled.
The Foo(Bar(Baz))) calling style is of course equally broken.
Parinfer solved this problem for me: https://shaunlebron.github.io/parinfer/
Editors too.
In Clojure I miss the preemptive VM support for the Erlang "processes", the pattern matching and the elegance of the pid mailboxes.
In Erlang I miss the fantastic built in data structures, the platform reach (targeting the browser for example), the syntax (refactoring clojure is a dream), macros, the REPL, the tooling, the time constructs (refs, agents, atoms) and a bunch off more esoteric stuff. (I don't miss core.async cause Erlangs processes are much better).
I tend to use Clojure by default and Erlang when the problem particularly suits it.
I've also started turning all the special character noise down in all languages now. I'm usually trying to read the words, not all the cruft.
And I had the same thought as back then, too: "Concepts, Techniques and Models of Computer Programming" also used this approach, except it's only after a few chapters that they teach you "oh by the way, Oz also supports for and while loops" whereas everything until now used recursion. All my eng school buddies found that off-putting, but I always thought that if you put the book in the hands of a completely newcomer to programmer, they would never even think of it.
(I know SICP predates CTM by decades and may not be considered in the same league, and that Mozart/Oz is rather more obscure that any lisp/scheme ever can be, but I consider it a fairly important book as well, and one of the best textbooks I've ever read: very well structured, well written, very complete, starts shallow but goes very deep and very wiiiiidddeee in terms of knowledge.)
Extra nice since I first saw the article in the chat with the HN bot on Telegram.