The Emacs Problem (2005)
sites.google.com
sites.google.com
The other thing is that the core of Emacs is a masterful piece of domain-driven design, where the domain is "text editing and windowing". Its conceptual model is clear, simple, and consistent. (Some of the later additions don't have this quality, but fortunately they mostly stay out of the picture.) It has a rich language for its concepts—'buffers', 'text properties', and so on—and uses that language everywhere through documentation and code. This tremendously eases the burden of writing programs that interoperate, because everything is based on the same conceptual model and thus makes sense in the same way. Not all Emacs programs follow identical conventions—far from it—yet it's amazing how close they get, given how many there are. It's Emacs' exemplary domain model, as much as its Lisp character, that creates this conceptual unity.
So while Emacs rightly gets a lot of credit for the technical aspects of its extensibility, we should hear more about the design aspects. Its core design is a masterpiece and ought to be studied as an example of the power of software design itself—something we're mostly still pretty bad at. It might be hard to get that taken seriously, though, since on the surface Emacs is obtuse, clunky, and old. Only when you dive underneath does it become orderly and beautiful.
This touches on the core problem with the whole Java ecosystem. The existence of so many tools demonstrates, first, that the underlying technology is insufficiently expressive and, second, that many programmers would rather create tools than applications.
Leaving the second aside, the kinds of tools required for Java development are largely unnecessary in the Lisp world because the language itself is expressive and extensible. You don't need different representations for data and code, separate parsers, and frankly bizarre build systems (coughmavencough).
As ESR, I believe, said "Lisp is worth learning for the profound enlightenment experience you will have when you finally get it." Unfortunately, it also makes you more aware that things don't have to be as bad as they are in the Java world.
Hopefully s-expressions will be the next aspect of Lisp rediscovered in "modern" programming languages.
cache {
ttl = 1024
}
cache {
redis_server = "123.456.789.012:6379"
}
server = getRedisServer(config.getString("cache.redis_server"))
server.put(key, value, ttl=config.getInt("cache.ttl")
Java does have fugly enterprisey frameworks. But it's silly to claim that this is some property of Java.
Incidentally, Python also does "code as data as config" also - see django's settings.py, for example.
Recompiling a Config.scala file is fast anyway unless you change the type signature.
If you wanted to do things in equivalent lisp style, you'd just compile a Config.java/Config.scala file and add it to the classpath. That's only a few extra lines in your ansible script. Your configuration will be statically typedchecked - failure to include a required parameter or including a param of the wrong type will result in a compile failure.
This is unconventional but a very reasonable way to go. At the moment I'm just using typesafe config because it's more or less the standard, but the more I think about it the more I like the idea of compiling my config.
That's really not equivalent though. With Java and Scala, there's a sharp divide between what's code and what's data. Even if you create a Config.java class, you can still point to some bits and say "that's Java code" and other bits (like primitives or array literals) and say "that's data". Java doesn't know how to treat the code bits like data, or the data bits like code.
This means that there's a whole load of cool stuff that you just can't do with your Config.java that you could with a config file holding Lisp code. For one, you couldn't use macros to easily pre-process your config file. Or, as Yegge pointed out, you couldn't just turn the tag names (in an xml like structure) into functions that transform themselves.
Newer versions of Spring use Java for configuration. As you said, it's pretty cool to be able to compile and type check your config. As cool as that is though, it's not at all the same as putting your config in Lisp code.
Consider, if you were to do this the way you are describing. How would you support saving updates to the configuration from within the application?
Elegance seems to be much more highly valued in the Lisp world than in the Java world.
I think it is something that is bound to the enterprise architect culture.
This leads to enterprise software vendors trying to write extension/customization languages for their tools. And of course, they almost all do it badly, and they all do it differently. (If you're hearing an echo of Greenspun's tenth law here, well, yes, I think that this is exactly what he was talking about.)
Enterprise architects have a surprisingly big diversity of culture. I'd look first at the process by what enterprises choose their development stacks. It often has little input from architects, developers, testers, or admins.
e.g.
./data.py:
me = "RamiK"
EOF
./code.py
import data
print(data.me)
EOF
Still, elisp is elisp is elisp and there is no getting around that fact. The last Common Lisp rewrite/transformation has, AFAIK, stalled. 'cl is still not welcome in core packages. Just staying with the language and dealing with its quirks has proven very effective and I think this is in a large part due the flexibility of being a Lisp. It might be the wrong language for text-processing, but is the right language for ancient software that tries to stay recent.
Thrill me.
Startup times less than seconds.
The dead simple ability to execute any lisp in any editor with a few key presses.
A really good integration with REPL environments.
A somewhat nice way to treat a shell as a REPL.
A mail client I like. (Hard to say that is good for others...)
And, it is overstated, but org-mode really should be looked at. Literate Programming. Task management. It has it all.
Registers. Each kind has its own use.
Painless keyboard macros. Name, save, edit, insert a counter, make them yours.
Dired. Wdired. The find-*-dired functions.
Org-mode.
Tramp.
These are all built in. Beyond that: a major mode for pretty much any language you can think of. Packages like magit and expand-region also immediately come to mind.
One interesting library I've come across is skewer-mode. It depends on a package that implements an asynchronous web server to inject some handy javascript into your browser that opens up a two-way channel with Emacs. I just have to add the href to load the script in my file and then I can open up a javascript REPL in emacs and load code into my browser without refreshing or anything.
I've heard of projects that let you programmatically embed and control webkit instances.
I publish books from emacs.
It's a handy programming environment and not merely a single-purpose application.
Eclipse's uses XML to declare UI, behavior, expressions. Completely baffling. Terrible docs. Extremely difficult to debug. I was reduced to trial and error (after copying working examples I found).
I've used Eclipse since the early alphas. Its muscle memory.
Scouting the plugin dev stuff of IntelliJ, it doesn't appear to suck so mightily. I've forced myself to start using IntelliJ.
I miss LISP so much I could cry. Even so, I remain wary of metaprogramming. And I very much want to do plugin dev, vs modifying the runtime, for the separation of concerns. I've had to customize all-in-one runtimes and borking the whole runtime with a bad override is not fun to debug.
I've not done emacs plugins, so can't comment or compare.
Your concerns are artificial! Let go of your concerns and be free. The hundreds of Emacs users around the world are already doing it. It's great.
It used to be a bit of a wild-west without lexical binding but we have that now and it's going over pretty well. Fewer libraries step on each others' toes as adoption spreads.
We have package management built-in. We have awesome introspection and debugging tools. There are a lot of wonderful built-in packages.
My init file isn't even written in raw elisp anymore. I use a well-documented org file. The amazing org-babel library is bundled in. My init file is written for humans to understand and only incidentally for my editor to execute. It's great.
I only wish more software was designed this way.
Came here to say that I do this, too: https://github.com/nhoffman/.emacs.d
Not sure if it was worth the effort, but it was fun!
http://sachachua.com/blog/2012/06/literate-programming-emacs...
What Emacs gets by being an lisp based system, is a pervasive dynamic circularity that, Eclipse (among others) ~is far from enjoying[1]. Anything is a burden in Eclipse, the SDK is a maze of stateful randomly order objects[2]. You needed a book to have a menu action running your code.
In emacs it's one line, very declarative, very composable[3] and you're done. It's pretty and gets out of your way. Everything in Eclipse was over-engineered (plugin repos that takes minutes to actualize, so much for Java jitted static typing performance). The only joy you get is the surprise you actually got the elephant dancing.
This and the target miss. It was a reinvention of Visual Age but it lacks good user interaction basics (edition is very crude, hard to extend, snippets were a new shiny thing in 2007)
And now that elisp has lexical-scoping, you can enjoy functional idioms to the fullest (dash.el) making your eclipse plugin java code (prior 8) even harder to swallow.
[1] at least when I was quite messing with Eclipse (circa 2007) EMF, OSGi, Eclipse plugins, Eclipse SDK running in Eclipse.
[2] how many ways you can get a particular object, accessing things through indexes (myArray[0] is not a stable path to anything)
[3] write a simple function, wrap it in a command, wrap it in a menu or a keybinding. very iterative, layered ..
But design wise, the Eclipse plugin system is awful comparatively. Some of that comes from Java's native lack of extensibility and static nature. Some of it is just plain old design pattern profusion.
I wasn't an org-mode believer until a few weeks ago, and then used it to bookmark and track notes for a large software project I had to audit, and it was amazing.
Remember mode is just a very simple way to remember stuff, and it can be plugged into org mode.
The biggie that I forgot is calc. Just awesome.
Lists http://www.gnu.org/software/emacs/manual/html_node/calc/List...
Algebra: http://www.gnu.org/software/emacs/manual/html_node/calc/Basi...
Algebra rewrites: http://www.gnu.org/software/emacs/manual/html_node/calc/Rewr...
Programming calc: http://www.gnu.org/software/emacs/manual/html_node/calc/Prog...
And (gasp) Data Types: http://www.gnu.org/software/emacs/manual/html_node/calc/Type...
(mind you, i mostly use the HP-calculator level stuff. But the rest is certainly awesome)
https://developer.mozilla.org/en-US/docs/Web/API/CDATASectio...
If you have more data than text, use JSON. If you have more text than data, use XML/HTML.
Imagine HTML documents encoded in JSON -- in that case JSON would be ugly and annoying to edit. Now imagine (the more common situation of) structured data serialized as XML. That is ugly and annoying to edit.
Use the right tool for the right job. :)
<Foo>Some text <Bar>more text</Bar></Foo>
{
'tag': 'Foo',
'children': [
'Some text',
{
'tag': 'Bar',
'children': ['more text']
}
]
}
['Foo', ['Some text', ['Bar', ['more text']]]]Also, side observation, that Lisp-like encoding is pretty icky as-is (deciding if something is a tag or text by how deep the array is?), and gets worse if the XML uses attributes.
'(foo "Some text" (bar "more text"))
I don't think your second example is much at all like Lisp and I think to actually achieve something lispy, JSON would need some kind of symbol type, so you can do:
['Foo, ['Some Text', ['Bar, 'more text']]]
After all, it's not like:
'(Foo "Some text" (Bar "more text"))
'(Foo "Some text" (Bar ([] (name "Baz")) "more text"))
<Foo>Some text <Bar name="Baz">more text</Bar></Foo>
One can write a parser for the canonical representation in less than an hour, and a parser for the advanced representation in a weekend.
They are infinitely nicer than XML.
The other thing that I found interesting in this article was the reference to regular expressions as being really useful for text processing but not good for handling tree-based data and I very recently found that the Stanford Parser has tree regular expressions/tregex[2].
[1] http://en.wikipedia.org/wiki/Outline_of_natural_language_pro... [2] http://nlp.stanford.edu/software/tregex.shtml
"Doing this effectively requires your data to be tree-structured."
I have actually been replacing XML with YAML in lots of places. I find the structure much less verbose, and easier to understand, and has features that JSON lacks, and I almost never have problems with delimiter collision. If you haven't yet, I really suggest checking out YAML, but that's just my two cents.
Also, "plugins are super-easy to write" is not much of a benefit to someone who wants to, you know, actually write apps rather than write plugins for the editor in which they will ostensibly one day write an app. Imagine if someone told you to use app X instead of Photoshop because the plugins are easier to write than Photoshop plugins.
You'd be surprised. They might have clunky editors, but they're far more advanced in dealing with code intelligently (refactoring, AST parsing and transformations, knowledge of structure etc) than ST or Emacs.
And now anyone with access to a "text" file can run arbitrary code. Or am I missing something?
This is how you can tell that madness lies that way.
Most other languages don't have an obvious way to just parse a file of source code without actually executing it: `#include <stdio.h>` or `import mycoolpythonlib` or `require __DIR__ . '/goolib.php'` both parse and execute/compile/link the file, so yes, any config/log file becomes a code file and "arbitrary code" gets executed, but it doesn't have to be this way.
(Now, you have things like "import hooks" and access to the AST in Python for example if you want a tree of the imported file, but nothing "simple" and obvious. And this is the thing actually: as much as people consider Lisps too complex and powerful, the nice thing is how "obvious" and "simple" they make things like this - by just parsing sexp tree and manioulating nested lists instead of inventing a file format like XML with its transformation languages, query languages and so on, and all the tools to support them, and all the man-years (or is it decades? ...centuries?) spend to code and maintain them ...just think how much needless effort could have been spared.)
Practical Common Lisp is what grabbed me finally (it's available for free online http://www.gigamonkeys.com/book/ or you can buy the print), and you'll love it if you more "to the point" and practical examples.
I get that reading in an log or config file as an S-expression is kind of cool, in a "one ring to rule them all" sort of way, but... what's different between that and reading it in as a DOM or SAX tree?
Is the claim that Lisp's reader is somehow better than DOM or SAX? Easier to use? The resulting trees are easier to walk? Easier to use? What's the actual improvement from doing it the Lisp way?
It seems to me likely that Lisp can import a file into a tree easier (on the coder) than SAX can, but not significantly so. It's not one line versus dozens; more like one line versus a more complex line or maybe two lines.
I argue that the point should always be to reduce complexity of the systems and keep them DRY, even at the cost of slightly less functionality and slightly worse performance. Mainly because the introduction of complexity in a system tends to be irreversible in practice (whereas performance can always be better tuned and more functions can be added when/if needed).
Now, in the real world, where the complexity of XML and friends (and this is probably not even the best example) has already been introduced, and where you can't make it disappear because you have so many things that depend on it now, you're right, there is no real benefit of the fact that you can also do it in a simpler way. And most pro-Lisp arguments hit the same walls: yes, you have a more elegant way to do things, but the complexity of the other ways of doing the same thing has already been introduced and it's not going away, so there's usually little benefit for also having simpler ways to do things.
Maybe you made the impression of a smart ass interviewer who likes to make up retarded interview exercises, and your interviewees played along. You might want to work on that.