The GNU extension language
wingolog.org
wingolog.org
I wonder, though, whether some things that made Emacs/Elisp so successful would get lost in the switchover. For example, while dynamic scoping is bad for writing robust complex software, it does make Emacs easier to extend.
Emacs is not really an application or a platform, it is a framework: a framework for a text editor, one that comes with a well-thought-through model and a flagship application (Emacs itself) that doubles as a development environment. I don't mean "model" as in MVC, but in the sense of a consistent set of concepts that can be composed to do things. In Emacs those concepts are things like buffers and windows and text properties. Emacs has one of the best models I've ever seen; it should be taught as an example of how to do this kind of design. Emacs on the surface may be clunky and unlovable, but dive underneath and its world is beautiful.
When people extend Emacs, they're building programs out of that model using tools designed for building programs out of the model (which themselves were built out of the model). If you shift from Emacs to GNU as a whole, you're moving from that single -- self-consistent and composable, not to mention interactive -- design context into a hugely diverse environment with no common design context at all. If Emacs is a planet, GNU is a universe. It's not clear how well Emacs' extensibility magic can be reproduced at that level. If the benefits come mostly from the language, Andy may be right. I hope he is. But if the benefits come mostly from Emacs' framework and interactive environment (both of which are inseparably bound up with the language), then the magic is likely to be left behind with them.
Still, it's hard to argue against his case for choosing the most powerful possible extension language so as to maximize the extending that people will be able to do with it. To argue against this in favor of familiarity is particularly to miss the point.
Interestingly, from what I've seen dynamic scoping hasn't hindered emacs lispers at all. They have some sort of variable naming system that is specific enough that confusion is fairly rare (I've never seen anybody talk about variable naming conflicts, at least). For example, the value of the (chosen randomly out of my .emacs) ``ido-auto-merge-work-directories-length'' variable seems unlikely to be changed accidentally.
That being said, Scheme/Guile have dynamic fluids which allow for the same sorts of things to be done. Personally, I feel like these would have to be used heavily in order to for a hypothetical lexically-scoped guile-based emacs to work, but maybe thats just because I feel like dynamic scope makes things easier to extend :p (at least, in this sort of program).
flet and flet*:
---------------
These constructs behave exactly like let and let*, except that they bind
the function slots rather than the value slots, and so make dynamic scoping
available for functions, too.
[1] http://git.savannah.gnu.org/gitweb/?p=guile.git;a=blob;f=mod...
It gives you easy access to everything that's going on at runtime. For example, many Emacs buffers are read-only or have read-only bits of text (e.g. the prompt in a Slime REPL). If your code tries to change that text you get an error. But there's a trivial idiom for getting around it:
(let ((inhibit-read-only t))
(do-the-thing-that-changes-text))
Because Emacs is dynamically scoped, everything works this way. You don't depend on the author of some module having said in the past, "Someone might want to modify this variable at runtime someday, so I'll declare it to be dynamic." Everything is readable and writeable by default. This is a big deal for Emacs' extensibility.
Dynamic scoping isn't good in general, but Emacs has benefited handsomely by making this tradeoff in its own case. There are thousands of useful Elisp modules that interoperate with remarkably little fuss, all extending the same system. Yet the language is dynamically scoped and has zero encapsulation. Most people, I think, would deny that that was possible if it didn't exist. Many people still deny it is possible even though it does!
When writing e-lisp, I find dynamic scoping limiting, and using lexical-let, especially when I want curried/partial-application, is extremely annoying.
For me, extending emacs is harder because of dynamic scoping. It is entirely possible to have code be executed in some sort of "Reader" monad/env, and execute it in a different than default env -- which gives you "dynamic scoping" on whatever input you want, without contaminating every other name.
I think almost every negative feature you could possibly implement would find "good use" that defended its existence (Dynamic scoping, $_ in Perl, implementation inheritance, etc).
Macros make it too easy to define new syntax, to the point where things that should be core language features are neglected, only to be continually re-half-implemented, etc. Take a look at what passes for 'good enough' for destructuring in Clojure, both the older let-based and the newer core.match (which still lacks the extensibility of Racket's match). The deficiencies can always be fixed 'later', right?
No syntactic difference between function application and special forms means that one requires an editor that can somehow differentiate (slime/drracket approaches) to even have a shot at quickly scanning unfamiliar code (say, quickly looking at what an implementation actually does). It's pointed out that Guile will handle other languages (imho this is tacitly admitting uncertainty about Scheme). But multiple languages only stand to make the code-reading problem even worse! Never mind that Scheme will be the only first-class language (other languages can't consume macros) so alternatives are at best advertisements to get people hooked and having to learn the 'real language'.
The little bit of Ruby I've written opened my eyes to how much you can do with just first-class code blocks, and made me understand why something like SLIB would opt for explicit lambdas everywhere instead of macros. Macros are so enticing to cut down on boilerplate, but they're visually indistinguishable, and without plenty of forethought they're not composable.
Clearly many skin syntaxes for Lisps have been attempted and never caught on. I don't currently have much to add on that front besides a suggestion of not being afraid of well thought out concrete syntax (how many base special forms are there, really?). But really, I think my whole post shows the main problem with Lisp, at least for me. It's so easy to do things and it's such a blank slate that no matter what I sit down to do, I always end up trying to create a new language.
The problem with this argument is that a sexp-based configuration language is already adopted by a community of programmers: Emacs Lisp. Emacs itself ships with over a million lines of Lisp code, and there are thousands of extensions available from around the Internet.
The problems you worry about with respect to macros have simply not happened; most people name macros the same way (starting with def or define-), and most macros are written "cleanly" in such a way as to not surprise the programmer. (I'll be honest: I can't think of one time I've ever hit a but in a Lisp program caused by a strange macro.)
Emacs Lisp shows that if you give someone a turing-complete programming language on top of a powerful piece of software with a clean and well-documented API, programmers will ignore the warts and enhance the software in ways you never thought possible.
Guile is not a perfect programming langauge, and I would never use it by choice. But if I started using some piece of software that used it for configuration, I know I would love it because it would stay out of the way and let me make things. Once you are making cool things, you don't care about the syntax or programming language features at your disposal: you just make it work.
Some other examples of this include JavaScript for Firefox extensions and Vimscript for Vpim extensions. JavaScript is very ... minimal compared to other languages like Perl, Python, Ruby, Haskell, and so on, but people write great browser extensions anyway. Vimscript is the weirdest thing I've ever seen but Vim is able to do a lot of cool stuff despite this.
The only thing a successful extension language needs is something worth extending. If you use Guile as your extension language, you can concentrate doing that instead of writing your own programming langauge toolchain.
What about macros that have the same name? In Emacs, the convention is to name your functions foo-this and foo-that if your program is foo.el. But this does not seem to be true for macros. Is that because macros don't leap outside the compilation context of foo.el? (Edit: nope, I checked it and they do.) Or is it that people are sloppier about naming their macros?
Edit: or is it rather that, if your foo.el defines a macro 'blah, and I write bar.el that says (require 'foo) and then defines its own macro 'blah, it's my macro that gets used to compile my code, so the one in foo.el can't do any harm to it?
If someone already has a macro with this name, then don't call it that.
The reason for ignoring the normal package-function naming convention is that it's simply ugly to write code like:
(eproject-with-current-project "foo" (whatever))
(with-current-project "foo" (whatever))But I still think Lisps impose an overhead on reading code, and that overhead is highly unfriendly to a moderate GNU user who just wants to start poking around to see what happens under the hood and customize a few things. If a goal of the GNU project is to empower users to extend their own software, that burden should be avoided.
The problems I said with macros not being visually distinct reflect what I've personally found when trying to grok others' code. Many special forms are easily picked out, but one must be sure that all have been picked out in order to separate the control flow from the expression trees.
Interesting thought, but where is your evidence? From personal experience, I've spent the most time reading and writing Perl and Emacs Lisp. As a result, I find both of them easier to read than Python, which is a language designed to be readable. The reality is that all computer programs are hard to read if you don't practice reading them. (As an aside, I would estimate that about 20% of programmers "in industry" are able to fluently read code. The rest just write the same thing over and over again and are confused when they are asked to fix bugs in code they didn't write.)
Does XML count as a "sexp-based" language?
XML kinda showcases another point.. Give someone the tools to whip up a 'language' without really thinking about their underlying problem and incidental complexity abounds.
"Robert says:
30 August 2011 11:55 PM
It isn't whether Guile is good, bad, usable, or not usable. It is whether people will and do USE it. I have not seen that as the case for Guile or Scheme in general."
I don't know anybody anywhere who has even tried to use Guile. I don't even know if anybody I know have ever looked at it. I don't know how many people even know it exists.
I personally love Scheme as a language, but I honestly can't see myself ever investing enough time into Guile to be able to write program extensions in it. I don't see my friends (some of whom I believe would write extensions to popular open source software if they could do so in a language that is sufficiently similar to one they already know - I certainly would be a lot more likely to do so and I'm not even Lisp averse) doing so either.
"First of all, what does it mean to have a GNU system based on a non-GNU technology? What would be left of GNU?"
So pick a non-GNU language and create a GNU implementation.
Personally, I think the best choice would be Lua. Its mature, a lot of people already use it as an extension language and its very similar to a lot of popular languages, so resistance for people to learn it is low (it is reasonably similar to Javascript and to a lesser extent, Ruby and Python; also, as has been proven by the game industry, it is easy to learn for C, C++ and Java programmers. Finally, while not GNU, it has some exceptionally good implementations).
[1] http://lilypond.org/doc/v2.14/Documentation/extending/scheme...
This summarizes it nicely.
I think the issue with the limited number of extensions written in Guile is more with the projects that can be extended than with the language--I'm not aware of any that I use, and certainly none that I would like to extend.
[1] by average I mean that they have the technical skill to do so, but aren't terribly invested in learning lots of new things. I imagine (I'm a vim user myself and I find its true for me) that an emacs user has already invested in the tool enough that learning a little more to make it even more useful is not a big deal.
The sort of person who doesn't go out of his way to learn a new tool is also the sort of person who probably doesn't go out of his way to extend an existing tool either. I think targeting advanced users for writing extensions is completely reasonable.
Plenty of "average" (by your definition) programmers who use Eclipse know Java well enough to write Eclispe plugins. How many of them do? I suspect a large portion of those who do would have written the plugins in Guile as well.
[1] I did actually write a few Python scripts to generate repetitive Java code because I felt that doing so would be much less effort than writing the code by hand.
I would argue that overall[1], guile has had pretty much no meaningful adoption in GNU projects. Emacs is (of course) still using its own flavor of lisp. gdb, which badly needs a powerful scripting language (and which I think was scheduled to have Tcl support when the wars started), still has no good scripting integration. I cannot think of any other major GNU project using guile.
[1] There are certainly some exceptions. Lilypond uses guile extensively and successfully.
I argue that Erlang parse transforms are at least as powerful. Interesting that he mentions Erlang, but doesn't mention that! (Erlang probably isn't ideal as an /extension/ language for other reasons, though)
If Lisps macros are difficult for the average user to learn then Erlangs Parse Transforms are even more so and a much larger barrier to usage than lisps. It's rare for an erlang developer to reach for a Parse Transform it's rare for a lisp developer not to reach for one.