FemtoEmacs: Tiny Emacs clone with configuration in FemtoLisp | Hacker News Reader

FemtoEmacs: Tiny Emacs clone with configuration in FemtoLisp

github.com

44 pointscm334 comments

Aside: femtolisp is used in the Julia parser, and was created by one of (or the) main Julia contributors. A femtolisp REPL is included in the Julia executable:

   >> julia --lisp
  ;  _
  ; |_ _ _ |_ _ |  . _ _
  ; | (-||||_(_)|__|_)|_)
  ;-------------------|----------------------------------------------------------
  
  > (+ 1 2)
  3
  
  >

As an answer to sibling comments: if your Emacs clone doesn't support elisp, then it's not a replacement because the elisp code out there is the #1 selling point of Emacs. And much of that elisp code exists because the editor is a flexible core with stuff exposed to elisp and all kinds of crazy stuff built on top of it.

Some numbers: there are ~120kloc of Elisp in Emacs proper. I have nearly 500kloc of Elisp in my `.emacs.d`. Granted, some of it are reimplementations and wrappers around what is in the core, and some are plugins I downloaded, tried out and forgot to delete, but still, that's a huge amount of code. That's 136 man-years, according to sloccount:

    Total Physical Source Lines of Code (SLOC)                = 499,088
    Development Effort Estimate, Person-Years (Person-Months) = 136.18 (1,634.17)
     (Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05))
    Schedule Estimate, Years (Months)                         = 3.47 (41.59)
     (Basic COCOMO model, Months = 2.5 * (person-months**0.38))
    Estimated Average Number of Developers (Effort/Schedule)  = 39.29
    Total Estimated Cost to Develop                           = $ 18,396,178
     (average salary = $56,286/year, overhead = 2.40).
    SLOCCount, Copyright (C) 2001-2004 David A. Wheeler

>As an answer to sibling comments: if your Emacs clone doesn't support elisp, then it's not a replacement because the elisp code out there is the #1 selling point of Emacs.

For some. For others a clone that does 20% of what Emacs does and has a few nice plugins could be enough.

My favourite Emacs clone is mg [1] from OpenBSD, but packages exist for Linux as well. I've cross-compiled it to many ARM embedded systems. Earlier, I used to use Zile, but they moved to Lua which made cross-compiling harder (so gcc / binutils are no longer enough).

[1]: http://homepage.boetes.org/software/mg/

Lua has great support for cross-compiling. It is written in the subset of features available in both (ANSI and more recent) C and C++.

https://www.lua.org/manual/5.2/manual.html

Not denying that. Lua by itself is easy. The thing is, to compile Zile, you need to first x-compile Lua, then x-compile luarocks, install the necessary Lua packages, then continue with x-compiling zile.

Comparatively, I've found that x-compiling vanilla emacs `--without-x` is often more easier and faster.

X-compiling Mg is more or less "gcc *.c" which is very straight-forward.

I've been using JED [0] (programmed in S-Lang [1]) for over a decade. It's my go-to editor if I'm at a terminal on some remote machine. I build it (it's pretty portable) if I need it, and I get (either out of the box or because I configured it) all of the emacs keybindings that I can't live without.

I don't use many advanced emacs features; this only covers basic editing (moving around, cutting and yanking, and macros) but it's the smallest subset I need to be productive.

I should look into this. I wonder how it compares.

[0]: http://www.jedsoft.org/jed/ [1]: http://www.jedsoft.org/slang/

> if I'm at a terminal on some remote machine

Why not run Emacs on local machine with plink-ssh to remote?

I really dislike editing files remotely. I'm usually dropping into the shell to run commands, change my directory, and I work much better if I can pop up a terminal where I am, quickly edit a few things, and exit. It's just my preferred workflow.

In my experience as a hardcore emacs user (10 years already of using it daily for almost everything) learning to use vim (evil in emacs for the past 3 years) has made me use the command line differently for small edits. Before it was nano, or remote emacs, now I can handle basic vim just fine to do the quick edits I may need on a remote machine

> I'm usually dropping into the shell to run commands, change my directory

TRAMP enables this pretty well …

Femtolisp looks interesting, although the weak documentation makes it hard to know. It it has no lexical scoping, no TCO, and no hygenic macros (I don't care if it's lowlevel or highlevel (er-macro-expander, or syntax-rules), or even if it's fexprs like Kernel, but it has to be hygenic), I tend to dismiss it out of hand, unless it has some really cool feature that means it can't, or is otherwise interesting (newlisp, picolisp). But femtolisp has most of that, so my only concerns are lack of hygene (which is a big concern in a lisp-1), and taking too much of the large and unweildly r6rs. But that would be hard, given the size.

According to the readme[0], femtolisp has both TCO and lexical scope.

[0]: https://github.com/JeffBezanson/femtolisp

Well, yes. That's why it was even in the running to begin with.

Ah. Your sentence should have read "If it has..." and reads "It it has.." leading me down the wrong garden path[0].

[0]: https://en.wikipedia.org/wiki/Garden_path_sentence

Yeah. I mostly do HN on my phone, which makes it hard to catch these sorts of errors, although I do my best.

Why does lack of hygenic macros matter? You can go from basic macros to Clojure's syntax-quote (quasiquote with auto gensyms for certain reader-defined forms) in 100 lines of code[1]. I've listed a few other implementations of the clojure-style of quasiquote plus auto-gensym, but there are a ton of them floating around the web for almost every lisp you can think of.

[1] As a clojure library - https://github.com/brandonbloom/backtick/blob/master/src/bac... (although you have to rewrite lines 51-56 to use explicit gensyms in a let instead of using the internal syntax-quote so that it bootstraps cleanly).

[2] Using the alternate Clojure reader - https://github.com/clojure/tools.reader/blob/master/src/main...

[3] Using Hylang - https://github.com/hylang/hy/blob/master/hy/core/macros.hy#L...

The problem isn't having to make sure that you've gensymed everything. I said I'd be okay with er-macro-transformer, from scheme, which has the same problems, albeit with proper hygene and with rename instead of gensym, which is nicer (and indeed, in CHICKEN we have ir-macro-transformer, which is like syntax-quote, but better, because it operates on the generated output instead of on the input). The problem is gensym itself.

Most people think that the only axis macro systems rest on is hygenic vs non-hygenic:

  Hygenic     |Non-hygenic
  syntax-rules|defmacro and gensym

This actually isn't true. Macro systems rest on two axies: Hygenic vs Non-Hygenic, and Lowlevel vs Highlevel:

  Hygenic Lowlevel    |Non-hygenic Lowlevel
  er-macro-transformer|defmacro and gensym
  ir-macro-transformer|
  sc-macro-transformer|
  --------------------+----------------------
  Hygenic highlevel   |Non-hygenic highlevel
  syntax-rules        |???

Highlevel macros use declarative pattern-matching. Lowlevel macros are imperative. Hygenic macros guarantee that there will be no name-clashes in a rename. Depending on your language, gensym may or may not do this, but usually does. However, hygenic macros also guarantee that all renamed expressions take on the value they have at in the environment within which the expansion occurs. Gensym does not, and never will, do this. This is acceptable in a Lisp-2 with packages (barely), but in a Lisp-1 of any sort, there is an extreme risk of function shadowing, and other nasty bugs (see http://community.schemewiki.org/?hygiene-versus-gensym for details).

Hopefully, this explains why I insist on hygenic macros, and why I think PG really screwed up by not adding hygenic lowlevel macros to arc.

I think the examples I listed (2 of them anyway) actually would be considered hygienic by your standards.

syntax-quote as implemented by clojure makes three guarantees:

(a) symbols in the body that are postfixed by `#` will be uniformly replaced by a unique gensym in the expansion

(b) unquoted things are left alone

In so far as is possible with a mutable function namespace (and clojure has this, you can alter the contents of vars and add/remove vars from namespaces), the syntax-quote in Clojure provides the environment within the expansion occurs as well as guaranteeing no name clashes (unless you explicitly opt into them, e.g. unquoting a quoted symbol).

If you wanted to do something similar to what the article mentions for common lisp, you end up at:

    (intern (ns-name 'clojure.core) 
            'symbol-you-want-to-replace 
            (fn [x] alt-definition-here))

I guess someone technically could do that, but if they do, I would rather my language err towards giving me the power to make bad decisions than hamstring my ability to do useful things.

In summary, as long as you have lexical scoping (which FemtoLisp claims to) or namespaces of any sort, you can solve all but the most contrived problems of something like syntax-rules without the additional complexity and restrictions. Again, this can just be added to the language by the user, if they really want it.

This comes back to a larger question of, if everything about a language is interesting except its macro system, wouldn't you be well-served to just bolt on the macro system that you desire? (Not saying that FemtoLisp is for you.) Seems that a hygienic vs non-hygienic macro system is fairly trivial if you care enough about the other features.

Syntax-quote still isn't actually hygenic by my standards, assuming you can do this (you can test). I don't have a computer in front of me, and I don't know clojure super well (like some of it, don't like some of it, HATE Java and the JVM):

  (fn a_func [x]
    (print x))

  (defmacro a_macro [y]
    `(a_func ~y))

  (let ((a_func (fn [x] (print "oh noes!"))))
    (a_macro "this is a string"))
  ;;;if I'm right, should print "oh noes!"

If this does work properly, and no variant of it works, than congrats clojure, you have hygene, or close enough.

>I guess someone technically could do that, but if they do, I would rather my language err towards giving me the power to make bad decisions than hamstring my ability to do useful things.

Which is why I prefer low-level hygenic macros, like er/ir/sc-macro-transformers: I CAN break hygene whenever I want to, but I have hygene when I want it, and lack of hygene when I don't. Despite what Doug Hoyte may say, I don't have my hands behind my back.

>In summary, as long as you have lexical scoping (which FemtoLisp claims to) or namespaces of any sort, you can solve all but the most contrived problems of something like syntax-rules without the additional complexity and restrictions.

What additional complexity? With ir-macro-transformer (not mentioned in the article), my macros are as clean as syntax-quote, and it's not that hard to learn.

Also, what additional restrictions? I don't see any.

>This comes back to a larger question of, if everything about a language is interesting except its macro system, wouldn't you be well-served to just bolt on the macro system that you desire? (Not saying that FemtoLisp is for you.) Seems that a hygenic vs non-hygenic macro system is fairly trivial if you care enough about the other features.

Well, femtolisp IS pretty cool, and if you hack together enough code, you can make a system that's almost hygenic, but I'm not sure if you can actually add hygene without the compiler's co-operation. Although gensym may be close enough as far as co-operation goes...

I would need to look into this further.

(defn a_func [x] (print x))

(defmacro a_macro [y] `(a_func ~y))

At this point, let won't allow you to rebind a_func (it changes the lexical scope, and the syntax-quote is going to refer specifically to ns/a_func). binding will also not allow it in modern versions of clojure, unless you declare the a_func var as ^:dynamic (e.g. ``` (defn ^:dynamic a_func [x] (print x)) ``` (as an aside, the compiler will also warn you for making something dynamic that doesn't have earmuffs, which is clojure convention)).

    (ns user (:use clojure.test))
    (def a 5)
    (defmacro inc-by-a [x] `(+ a ~x))
    (is (= (inc-by-a 0) 5))
    (is (= (let [a 0] (inc-by-a 0)) 5))
    ;; the is-macro implementation of thrown? doesn't catch
    ;; IllegalStateException...
     (defmacro actually-throws? 
        [exception & body] 
        `(try (do (do ~@body) false) 
           (catch ~exception t# true) 
           (catch Throwable t# false)))
    (is (actually-throws? java.lang.IllegalStateException
                          (binding [a 0] (inc-by-a 0))))
    ;; if we opt into this madness, all bets are off
    (def ^:dynamic a 5)
    (is (= (binding [a 0] (inc-by-a 0)) 0))

...so you're tying your hands by referencing any function from a macro.

I thusly conclude that Clojure's macro system, while hygenic, is inferior to most of the scheme macro systems, which would do The Right Thing in this case, without you having to append every function with a hash. Except for er, of course, where rename is explicit.

Your intuition that Clojure's macro system is unhygienic are correct.

Hygiene in this context was coined by Henk Barendregt in the context of the lambda calculus:

> For reasons of hygiene, it will always be assumed that the bound variables that occur in a certain expression are different from the free ones. [0]

This may seem nonsensical at first, but consider two different functions, f(x) and g(x): although both f and g take an x, they are in fact different variables. So, if some expression has x free, and another has x bound, and one is substituted into the other, you must remember to differentiate between the two homonyms. One way to do this is to consistently rename one or both variables so that they no longer appear to be the same, but it's important not to get too hung up on the concept of renaming per se, because the only requirement is that they be regarded as different (Andre van Tonder's delightful implementation of syntax-case uses a painting algorithm rather than a renaming one, but the net effect is the same).

In the case of syntactic transformers (macros), we're not concerned about the variables of the transformer itself, but rather the variables of the transformed code. Consider that full macro expansion occurs as a sequence of single expansions (transcription steps) until no further expansion can be done. Now, a macro is hygienic if all identifiers introduced during any given transcription step are unique to that transcription step (regardless of name). The introduced identifiers are analogized with bound variables while all other identifiers are analogized with free variables, just like how a lambda abstraction introduces its parameter variable.

Note that even unhygienic macro systems are capable of expressing at least some hygienic macros: macros that do not introduce any identifiers are trivially hygienic. I contend that the explicit-renaming macro system is not actually hygienic, even though it allows you to write any hygienic macro. The reason being that, in essence, hygienic macro expansion is precisely the notion of lexical scoping applied to syntactic transformation; and just as we don't need to manually specify the scope of each variable in our basis language, we should not need to do so in our meta language. For the same reason, Clojure's requirement to gensym identifiers -- while it offers convenient shorthand for the operation, it is still explicit (and it solves only one of the two problems of unhygienic expansion, namely variable capture, while providing no protection from shadowing). In that vein, syntax-rules, syntax-case, and implicit-renaming macros all fit the bill.

Just as dynamically-scoped variables can come in handy sometimes even in a lexically-scoped language (many Lisps have them, variously called "fluid variables", "special variables", or "parameters"), unhygienic injection of identifiers can come in handy sometimes (the classic example being anaphoric macros). The traditional way to do this is to simply circumvent, or "break", hygiene for a particular identifier, but Eli Barzilay and crew have also come up with the notion of "syntactic parameters" analogous to dynamically-scoped variables.

[0]: Henk Barendregt, "Introduction to the Lambda Calculus".

Neat. But er IS actually hygenic by the standard definition.

I'd say I'd check out what Eli &co did, but I probably won't. TBH, Racket leaves a bad taste in my mouth: the core is too complex for such little benefit. If some of it were in libraries, I wouldn't mind so much, but most of it's there by default. And I share Alex Shinn's opinions on syntax-case.

> Neat. But er IS actually hygenic by the standard definition.

It's hygienic in that it allows you to write any hygienic macro, yes. However, it's not "hygienic by default" -- you have to go out of your way to manually and explicitly ensure your er-macros are hygienic. Leave out a ,(rename 'foo) somewhere, and suddenly you've broken hygiene. Same problem with gensym, really, except that rename is a little nicer to use. Compare with syntax-rules, syntax-case, or ir-macros, where any macro you write is hygienic automatically and by default -- you have to go out of your way to manually and explicitly inject free identifiers.

> I'd say I'd check out what Eli &co did, but I probably won't. TBH, Racket leaves a bad taste in my mouth: the core is too complex for such little benefit.

It's also a proposed addition for R7RS-large (SRFI-139).

> If some of it were in libraries, I wouldn't mind so much, but most of it's there by default.

It's all in libraries. `#lang racket` is just a "metalibrary" that exports a bunch of smaller libraries. You can always do `#lang racket/base` and include the subset of libraries you need.

> And I share Alex Shinn's opinions on syntax-case.

Not to talk shit about Alex -- he's a smart guy and Chibi is a very nice system -- but he's both seriously misinformed about syntax-case and hopelessly obstinate about it. Despite his claims that syntax-case is super complicated to implement, it's really not much more complicated than syntax-rules: it uses the same pattern language, but rather than a template, it's associated with an expression. The only thing "extra" that's needed is a disjoint `syntax` type, which is just a wrapper around the other primitives with a bit of implementation-defined context (the crucial bit of context, for purposes of hygiene, being the syntactic environment; apart from API, a `syntax` value is really just a syntactic closure). The thing is, syntax-case has been shown to be able to express anything er-macros and synclos can express, but the reverse is not true, meaning that it's very possible that syntax-case is strictly more powerful than the alternatives.

I honestly think Dybvig and company did a disservice to syntax-case by making psyntax so complicated. I think one of their goals was to make psyntax production-quality, but I think it would have been better to make it as simple as possible -- just the bare essentials, for illustrative purposes. People tend to take it as exemplary because it's what they've got, but it really is a bad example because in its desire to be real-world and practical it obscures the intuition behind the system. Dybvig's entry in "Beautiful Code"[0] helps considerably, but I still maintain that the essence of the system is simpler than it's portrayed even there.

While I think that ir-macros are often very nice, there's no denying that the pattern matching offered by syntax-rules/syntax-case is extremely convenient and, more often than not, extremely clear. Many of the macros that are full of cadrs and caaaddddrs and cddaaars find a much more convenient and readable expression in the pattern language.

[0]: http://www.cs.indiana.edu/~dyb/pubs/bc-syntax-case.pdf

In your hygenic macro systems list, you forgot sc macros. Anyways...

>However, it's not "hygienic by default" -- you have to go out of your way to manually and explicitly ensure your er-macros are hygienic. Leave out a ,(rename 'foo) somewhere, and suddenly you've broken hygiene. Same problem with gensym, really, except that rename is a little nicer to use.

Well, I didn't say it was a great system, I just said if was hygenic, for the given value of hygenic.

>It's also a proposed addition for R7RS-large (SRFI-139).

Thanks. That actually does look pretty neat, although a bit odd in some ways.

>It's all in libraries. `#lang racket` is just a "metalibrary" that exports a bunch of smaller libraries. You can always do `#lang racket/base` and include the subset of libraries you need.

Well, there is that... But I'm still not a huge fan of Racket: I dislike where they diverged from Scheme (mcons? really?), syntax-case STILL leaves a bad taste in may mouth (see below), and the system as a whole just feels overly baroque to me.

>Not to talk shit about Alex -- he's a smart guy and Chibi is a very nice system -- but he's both seriously misinformed about syntax-case and hopelessly obstinate about it. Despite his claims that syntax-case is super complicated to implement, it's really not much more complicated than syntax-rules:

...I don't think that was Alex's objection to syntax-rules. It isn't mine. My objections are as follows:

-Breaking the standard macro abstraction by saying that expressions aren't, in fact, sexprs.

-Presenting an overy complex API to the programmer: lots and lots of functions, and a whole new read syntax.

-the (lambda (stx)) wrapper basically breaks every existing macroexpander, which, if it had a defined format in a given Scheme, probably used the format defined by sc. Why define that anyways?

Just a quick note to inform you that Racket now uses a new model for macro expansion. It is based on sets of scopes rather than the mark model used in syntax-case.

    http://www.cs.utah.edu/plt/scope-sets/

The new model has also been used by Tim Disney and others to make macro expander for JavaScript (see the sweet.js project).

At a glance, that looks a lot like that Syntactic Closures model: Modeling the environment as both mappings between variables and values, and identifiers and variables. However, I fail to see how renaming is problematic in recursive macros, save being unweildy. I'm not sure, but AFAICT, this may be a solution in search of a problem. It is interesting though, and might be an actual better solution. It's hard for me to tell without a long, slow readthrough, which may happen later.

But this is just a concrete instance of my opinion of Racket in a nutshell: Cool ideas, but sometimes overly complex for my liking, interesting systems that don't always mix well, a syntax over the ideas that I find distasteful on occasion, and documentation that is comprehensive, but hard to understand.

It's okay if you like it, and it's a good language for Getting S%#! Done, but it's not really my thing: I prefer Chicken: Simpler, less academic and cutting edge, but still really good at Getting S%#! Done. But that's my problem, not Racket's

For an editor a dialect of Lisp with proper module-based multiple namespaces (a-la Python) is a much better choice.

It is also nice to have a CL package, which makes porting of relatively easy.

And FemtoLisp is a beautiful thing, of course.

Maybe this can be the basis for what guile-emacs has promised but not yet delivered. If this gets elisp support, it could turn into a viable replacement.

The biggest issue with guile-emacs seems to be that the community doesn't seem particularly interested in it.

As a daily almost-everything-in-emacs user, I definitely understand the shortcomings of elisp, but generally speaking it does the job. Yeah guile would be an improvement, but I'm not sure it is worth rocking the boat.

It doesn't seem that the idea that elisp needs replacing has that much currency.

I meant replacing the VM, not the Lisp dialect. The value of Emacs is the elisp code available and the platforms it runs on. I don't think there are many users who like the underlying Lisp implementation's runtime behavior.

It would be nice if the releases were tagged in git. And if it could check /usr/share for the init stuff instead of $HOME.

Reply on news.ycombinator.com