Ferret – A free software Clojure implementation | Hacker News Reader

Ferret – A free software Clojure implementation

ferret-lang.org

266 pointsgreydius79 comments

It's very notable in that it's literate Clojure.

https://github.com/nakkaya/ferret/blob/0.2.8/ferret.org

Having written Clojure full-time for the past 5 years while writing cleancoders.com has really spoiled me. Clojure's introspective nature makes for some amazingly fast productivity because you can hook up your IDE to the REPL and have a kind of super-REPL, which can really speed up development a ton if you design your system it right. I'm not sure if this project can keep that aspect because of how it compiles to C++. But if it does, then it could really be invaluable in the Clojure ecosystem.

Author here. If I am not mistaken (I don't do web development.) very early versions of ClojureScript repl compiled code on the JVM side then piped the output to the browser. (Maybe it still does idk) I am thinking of doing something similar using Cling[1]. Ferret can compile and pipe output to Cling and use Cling as the repl.

[1] https://root.cern.ch/cling

There's a mooc on smalltalk on FUN. You should try it. You'll probably have a blast too.

Curious what IDE you have set up to do the REPL driven development you mentioned? I'm just starting out in learning Clojure and am finding various options out there with vigorous proponents of each. I am wondering what someone's real-world experience is after 5 years.

It's hard sometimes to find the editors/IDEs that might suite one particularly well since the vast community is attached to a given platform.

The most mature Clojure development environments right now are Spacemacs/Emacs with CIDER (free) and IntelliJ IDEA with the Cursive plugin (commercial, but inexpensive). Both are excellent.

I'd also add that support in Vim and Neovim are also very good.

Cursive is a very mature project nowadays. I'm surprised at how advanced it is considering how dynamic Clojure is.

Emacs + CIDER. It's extraordinarily fast, if you're willing to get used to Emacs :)

I use IntelliJ with Cursive, I have had to do zero configuration, it just works with lein projects.

I will eventually switch to Spacemacs and CIDER once I've really honed my Clojure skills, but for starting out Cursive was what allowed me to learn Clojure in the first place.

So far I've used vim, neovim, cursive and spacemacs and all have solid repl integration. I am generally a vim/neovim user and have been using that for years. It's good. Recently, I've been using spacemacs and like it better. Cursive is pretty solid too though if you're an IntelliJ user.

I would also like to plug Vim. I use it for both Clojure and C++ development, and it handles both like a champ.

Try Atom, following this admittedly opinionated guide. I think you might find it more accessible unless you're already familiar with one of the other alternatives.

https://gist.github.com/jasongilman/d1f70507bed021b48625

The only thing I would add is to try out smart mode in the printer package.

Discovering this and [Clojerl](https://github.com/jfacorro/clojerl) within a week of each other is like Christmas come early.

To tell the truth, I was expecting Perl, given the name.

I thought the same as you did when I first saw it and got a bit sad. I'm actually considering trying to implement something like Lisp Flavored Perl a la Lisp Flavoured Erlang as my senior thesis/project this year.

I'm kind of torn though, as it's my second real idea and my director already likes my first idea. While that one will probably end up being too much work, this one might be overcompensating for that (although I could just be overly optimistic about how easily I can write a new lisp).

Well, there's a few already implemented an on cpan[1], so even if you were being optimistic thinking you could do it without outside input, the truth is there's plenty of Perl specific prior art to examine which likely makes it quite easy to make a simple Lisp. I'm not sure how that maps to what you were thinking of.

1: https://metacpan.org/search?q=lisp

Oh, awesome, thank you. I hadn't looked on CPAN directly yet and general searches apparently weren't coming up with relevant CPAN results. CljPerl[1] is right along the lines of the sort of project I was thinking of doing.

I didn't specify, but I'm actually thinking of target Perl 6, so this is a great sort of halfway point where clearly other people have found it a worthwhile idea but not done exactly what I was thinking. Plus, P6 slangs could make it nice and easy to interleave regular Perl 6 and Perl 5 code with whatever monstrosity I come up with. I guess I have some decisions to make. Thanks again!

1: https://metacpan.org/pod/CljPerl

Just in case you've missed it: https://hy.readthedocs.io/en/stable/

I love Hy, but fyi, it is not as much a Clojure dialect as Ferret or Clojerl. The latter two share syntax and semantics, while Hy only shares syntax.

This is very interesting. I could really use a good language on microcontrollers, especially if core.async gets ported. Most microcontroller code has a huge (and complex) event-driven state machine at its core. Rather than making all of it explicit, I'd much rather use core.async to make it look like sequential code. It would really help with reducing complexity.

Does anyone have any insight on the legal status of this project? It’s licensed as BSD 2 Clause, but looks to be heavily derived from Clojure which is EPL. AFAIK the EPL is copyleft and doesn’t permit relicensing without the copyright holders’ permission.

Hard to say, would depend on whether or not they copied any code from the Closure compiler. It is possible they created their own implementation, but I wouldn't bet on it. Even if you treat it as EPL though, any output from the compiler won't be copyleft.

Author here. I am actually curious about this myself. I originally released it under GPL but then on another HN thread someone said anything compiled with it also ends up being GPL, so I switched to BSD 2 Clause. I have not copy pasted code from Clojure compiler BUT in order to keep the Ferret semantics the same as Clojure semantics algorithms are identical. [1] shows + function in Ferret and Clojure. As for data structures situation is similar, same semantics implemented in C++ instead of Java with some differences/tweaks geared towards embedded systems.

[1] https://gist.github.com/nakkaya/7050d5daf2c7034a25bb079658d8...

Nothing wrong with BSD - but as for a GPL compiler with a run-time, you could also look at the GCC/GNU libc licensing (GPL/LGPG). I do think it's important people are aware that GPL on things with a run-time should probably have an exception/use something like the LGPL - as you generally don't want "documents"/artifacts produced with a tool, to forcefully inherit the license of the tool used for creating them (eg: You don't want the word processor to force a license on documents produced, or the drawing application apply license to images - and the same goes for programming languages, IMNHO).

On another note, I'm not sure it would be quite possible to stretch "copyleft" that far legally anyway - but that's another discussion - and at ending up with an invalid license isn't really any good either, as that would leave you with just copyright and no license grant - making the tool illegal to copy/distribute...

LGPL isn't enough for a runtime unless your entire runtime is dynamically linked. Either GPL or LGPL with runtime exception works fine though (that's what libc and others have, I believe).

That is, they added an exception saying that if you use it in a certain way (unmodified, runtime for compiled code), then it's exempt from many of the GPL's clauses. Used in any other way, however, and you must comply with the (L)GPL

I'm not a lawyer but my understanding is that copying semantics could still be argued as creating a derivative work. This is what e.g. Whisper Systems claimed before publishing an axolotl spec, that anyone implementing the spec was doing so by deducing the semantic meaning of a copyleft piece of software and thus was a derivative work of said copyleft piece of software. This meant that the resulting binary was covered under the copyleft license (GPLv3 in that case, but my understanding of EPL is similar) https://medium.com/@wireapp/axolotl-and-proteus-788519b186a7

I don't know if Rich or the other Clojure copyright holders care enough to ever make trouble, but I'm pretty sure most companies with a decent legal team would think thrice before offering a piece of code built using Ferret

GCC has already solved the problem the other commenter you mention brings up with the runtime library exception https://www.gnu.org/licenses/gcc-exception-faq.html but I don't know if you can tack this on to the clojure-derived parts of ferret. I sure wish SFLC or someone was able to do legal counselling for new/small projects like this, I feel like a big reason people shy away from copyleft isn't the hippie dippie bullshit but because these interactions are impossible to reason about by laypersons

The favicon.ico of this site - looks exactly like half life logo.

http://ferret-lang.org/ferret-styles/favicon.ico

https://www.google.co.in/search?q=half+life+logo&oq=half+lif...

What is that all about...

It's the lambda symbol in a circle. Seems more like a coincidence than anything else to me.

There's plenty of ways to style a lambda. And plenty of colors to choose from.

That's the Half-Life logo, not a coincidence.

(And, just to prove how much I love Half-Life, my username is a Half-Life reference)

clearly they certainly respect half life a lot. maybe they don't know it exists actually. i don't know who made it because im lazy and didn't check, but I will say, there is certainly a non zero possibility of someone being alive today and never having played or heard of half life, even if only due to youth alone. but either way it is an interesting coincidence. I wonder if it's more than such.

Author here. It is the Half-Life logo. I wanted a lambda symbol for a favicon, love the game too so I settled on a open licensed logo.

I'll point out that the Half-Life logo is trademarked in the EU in several situations related to video games, including the programming of video games. Be careful attaching it to e.g. a tutorial on using Ferret to write a game, or an integration with a game engine, etc.

I wonder if it could also somehow lift the reactive synchronous concurrency bits from Céu[0] in a library. Céu is probably my favourite embeddded language to program in from a "thinking about concurrency" perspective. I'm not sure how one would go about it though, since Céu is imperative, and I'm not sure if that imperative aspect is fundamental to its type of concurrency or if it clashes with idiomatic Clojure.

If it would be possible you'd probably have everything I could want in one embedded language.

[0] http://www.ceu-lang.org/

Author here. Did not knew about Céu till now, just watched the intro video. I maintain a behaviour tree [1] library called alter-ego [2] (code needs cleanup) which already contains some of the constructs. In behaviour tree terms par/or is parallel selector and watching is a interrupter node. [3] So it can be done in Ferret on a general purpose PC using C++11 concurrency constructs, for embedded targets it will require co-operative multitasking which is on the roadmap.

[1] https://nakkaya.com/2010/06/29/alter-ego-a-reactive-ai-libra... [2] https://github.com/nakkaya/alter-ego [3] http://aigamedev.com/open/article/parallel/

> In behaviour tree terms par/or is parallel selector and watching is a interrupter node.

I see the resemblance, but I'm not sure if that's a perfectly right analogue. The description in your third link says the parallel code truly runs in parallel. However, Céu is concurrent but not parallel, making the concurrent code deterministic and easy to reason about: if multiple blocks in a par/-- construct await on the same event, they will execute in lexical order (which is similar to the behaviour described in the first link).

For real parallelism, you'd basically have to have multiple Céu programs (devices) communicate through external inputs/outputs.

I guess the real question I'm asking is whether the library follows the model of a synchronous programming language.

https://en.wikipedia.org/wiki/Synchronous_programming_langua...

BTW, the intro video is very outdated: on top of the basic par/--- and await construct, the "organism" is replaced with cleaner code/tight and code/await forms to create more complex abstractions:

http://fsantanna.github.io/ceu/out/manual/v0.20/statements/#...

Seems like a cool project if you already know Clojure and hate the start up time, but also seems like it sidesteps one of the main benefits of Clojure which is Java interop obviously... Which Clojure has to do a fair number of handsprings to accomplish. Begs the question, why pick Clojure as the lisp to compile to C++? Why not Racket or Common Lisp etc?

I like Clojure for its encouragement of functional, immutable-first programming. That, and I find its syntax for function parameters, data-structure literals, etc to be preferable.

Why not Racket? It has a similar emphasis on immutable, functional programming, it's also a Lisp-1 (I assume that's what you mean by "syntax for function parameters"), and the different syntax for data-structure literals I have to imagine is a pretty minor point.

I think they meant that function parameters are vectors when defining a function:

  (defn foo [arg1 arg2 & rest] ...)

As for data-structure literals, it's nice to be able to write

  [1 2 3] vs (vec '(1 2 3))

  {:key1 "val1", :key2 "val2"} vs (hash-map :key1 "val1" :key2 "val2")

Agreed, all pretty minor points, but all nice to have. Function arguments are easily distinguishable and creating maps/vectors is simplified.

I think clojure has very well designed concurrency and data structures. If this compiles to c++ presumably one could leverage the c++ universe.

Why not? It's great to have another Lisp for me to try out. I was avoiding learning Clojure specifically because of the JVM dependency.

I kind of saw it as being intended for someone that loves Clojure but needs to run in a constrained environment that usually requires C or C++.

Aren't most of these environments low on memory and thus it doesn't really make sense for them to use Clojure's persistent data structures? Kinda related is this awesome paper: https://blog.acolyer.org/2015/11/27/hamt/

I'm not sure how it's implemented in Ferret but Clojure's data structures are implemented as trees so there's structural sharing that happens when a vector or hash map is updated or when a new version is produced.

So the whole data structure isn't copied but only the relevant parts are added / updated in the tree and the rest is shared with the "new" data structure.

Mutable data structures are still more memory effective but I'm assuming its fine since the author has a blog post on using ferret on an Arduino Uno which has 32k of ram. https://nakkaya.com/2017/02/15/bare-metal-lisp-rc-control-us...

Clojure data Sturucture links: Persistent Vectors http://hypirion.com/musings/understanding-persistent-vector-...

Video with Rick hick on them https://youtu.be/wASCH_gPnDw?t=1742

It's so interesting watching Rich talk. He has such a breadth and depth of knowledge and you can tell that he really took his time when designing the language.

I always laugh when I see this complaints, given that Lisp was already running on IBM 7090, Univac M-460, IBM 360 and many other mainframes.

Any of those tiny environments run circles around the hardware constraints of 60's and 70's mainframes.

The relationship with Java is both a benefit and a disadvantage of Clojure.

> Why not Racket or Common Lisp etc?

These already have solutions in the compilation space.

Kyoto Common Lisp (KCL), a fairly old implementation tracing back to the 1980's, and its descendant GNU Common Lisp (GCL) compile to C.

Embeddable Common Lisp (ECL) also contains a Lisp to C compiler.

Of course, numerous CL implementations compile to native code without the C route.

Btw: ECL is also a descendant of KCL. Like AKCL, MKCL, Delphi CL, Ibuki CL, GCL.

Other Lisp-to-C compilers like CLICC, mocl, Thinlisp, and a few others are not descendants of KCL.

I'm looking at CLICC now a bit.

Boy, the implementors made a significant mistake in this one. Though CLICC is not intended to be a complete Lisp system, but a Lisp application, they insisted that CLICC has to itself be a CLICC-compiled application. And so what that means is that, then, programs being compiled with this bootstrapped CLICC are severely limited as to what they can do at compile time (in macros), simply because of the crippled run-time environment of the CLICC-compiled CLICC compiler which has to run without the benefit of an actual CL runtime. Whereas if CLICC had been kept as a Lisp-compiled program, only the run-time parts of user code would be restricted to the "CL1" language; macros could be written in full CL.

Clojure on JVM certainly benefits from Java interop, but ClojureScript compiles to JavaScript, and benefits from JavaScript interop.

For me, I like being able to take EDN and immutable data to the various platforms I need. My portability hurdle is little more than writing alternative functions where Java/JS interop doesn't port.

Congratulations, it's both very impressive and promising :) Thank you for that fine example of literate programming :)

I'd suggest you add a way for people to financially support you so you could keep working on this project.

I've been looking for a language which provides Software-Transactional-Memory with persistent data structures which fits in 2KB RAM on a microcontroller.

I doubt you'll find that quickly. You can't do much in 2kb if you don't want to mutate your data, I think. I don't know much about STM, but I'd wager that uses some extra bookkeeping memory as well. So unless your own memory requirements are really low, this won't work.

Fun fact: Clojure is one of the few languages in which STM is easy!

You're right in the general case - STM incurs significant bookkeeping overhead (at least a word per mutable location, usually more), which is one of the reasons it never took off. (The actual reason is the CPU overhead, which slows down memory accesses so much you've lost all the performance you might gain from parallelism.)

Clojure sidesteps this by having an incredibly small number of mutable locations. With a persistent data structures, every mutation operation already builds its own private version of the structure, and then atomically changes a pointer to point to the new version. So you've got MVCC built in, and you only need transactional metadata on one location (the ref).

He wrote about the motivations here: https://nakkaya.com/2016/06/10/ferret-a-hard-real-time-cloju...

Always look at the memory management story when you stumble upon something like this.

You could have provided a link to the memory management section or even mentioned it was ref counting. Otherwise your comment is ... it kills me to say this but pretty useless.

Always look at memory management is more general, and perhaps even more useful, advice than look at the memory management for this one project.

Why? What's "good" vs what's "bad"?

It seems well thought out:

* malloc/free (ref counting GC)

* memory pooling (no heap, stack-based for memory constrained systems)

* third party allocators

* third party GC

Details here: http://ferret-lang.org/#sec-4-2

I love the idea of ref counting, but I'm unsure about the handling of memory cycles. In a plain ref counting implementation, these will not be cleaned up, resulting in a memory leak.

I didn't find anything on cycles in the linked page, maybe the problem is not relevant in most real-world clojure code?

I don't know about "pure" Clojure (as this is), but in "pure" Erlang (which has similar semantics), you simply can't make a reference cycle. Functional languages in general don't really allow them.

Author here. As derefr said with Clojure semantics you can not create cycles.

Thanks, now I think about it, that makes a lot of sense.

Does this have Clojure's persistent data structures? The docs say it has immutable data structures, which isn't exactly the same thing. (Edit: added the second sentence)

Those persistent data structures are implemented in Clojure aren't they? It's not a feature of the compiler I believe. So if it can run Clojure code then that code can be the persistent data structures library if you want.

They are a feature of the compiler - Clojure's are implemented in Java.

And Clojurescript's are implemented in Clojurescript!

Viewing the site in iOS Safari, I get several pages of what appears to be random binary gunk rendered as HTML. Something wrong with compression headers?

Seems to be working fine for me on iOS. Any particular page that's broken for you?

The homepage, but just now I refreshed and it started working. Odd. (I’m pretty sure I tried refreshing before, and it didn’t work then.)

I smell an infoleak. What version are you on ? See if you can recreate it with your history. The flaw might be worth something.

Ohh embedded friendly, you're spoiling me.

This is amazing! I am wondering what was the motivation to implement it.

Would love to see a "Differences with Clojure" section.

Can anybody tell whether it supports multimethods and protocols?

http://ferret-lang.org/#outline-container-sec-7

The last item on the Roadmap underneath the "Core" Section lists multimethods.

Also, transducers and STM?

Reply on news.ycombinator.com