Second Climacs – Emacs Implementation in Common Lisp | Hacker News Reader

Second Climacs – Emacs Implementation in Common Lisp

github.com

131 pointsxvilka60 comments

That's very cool and I'm waiting for it but the usable Common Lisp based editors are Lem and LispWorks:

https://github.com/lem-project/lem/ supports CL, Python, C, Rust, CSS, Go, HTML, Java, JS, Nim, OCaml… (see the modes/ directory), it has a directory mode (à la dired) and it seems to support LSP (I only tried it for CL so far). It is very much Emacs inspired, but is lacking documentation and self-documentation features :( There is Lem-OpenGL ! https://github.com/gregcman/lem-opengl

The most functional, feature-complete, proven editor is the LispWorks IDE: http://www.lispworks.com/ A review: https://lisp-journey.gitlab.io/blog/discovering-the-lispwork... It's proprietary. The free version has annoying limitations. It is written with the portable CAPI framework. It has an excellent graphical stepper.

And, you might not know, but popular editors are getting good to very good support for CL now, at last. SLIMA for Atom nearly reaches feature parity with Slime (it doesn't have a stepper). For VSCode, see Alive. There's a Jupyter kernel, an Eclipse plugin and more. https://lispcookbook.github.io/cl-cookbook/editor-support.ht...

libraries: https://github.com/CodyReichert/awesome-cl

Cool stuff, Lisp is such a beautiful language and I hope more people get the chance to get to know it

The software we are talking about relies on McCLIM, whose github README.md says that "Right now the only backend supported by McCLIM is CLX, which ties it to the Xserver on the host system."

In contrast, Gnu Emacs has bindings to native GUI libraries on Windows, Mac and Linux. By "native" I mean Xserver is not required on any of them (and GTK is required only on Linux).

Details: on Linux the feature/pgtk branch of Gnu Emacs (of which I am a happy user) talks directly with GTK and Wayland with no need for an Xserver though there is a risk that feature/pgtk will never be merged into Emacs's mainline because influential Emacs users like to run their Emacs process on powerful computers remote from the computer in front of them, and so far no one has gotten that "remote" feature to work in the feature/pgtk branch without using "the TTY interface", which lacks features of the other interfaces described above (the Windows interface, the Mac interface, etc).

X has been the bane of McCLIM for the last 20 years. It limited adoption and delivered horrible first impressions to not-hardened Lisp users / McCLIM developers: Basic-level functionality that's supremely important for a positive experience such as font rendering, absence of graphical glitches, tearing, smoothness and overall look&feel was bottom-of-the-barrel for very long periods of time. Maybe it still is, I don't know since I no longer use X.

Hopefully that is about to change. Daniel Kochmanski seems willing to push McCLIM forward with alternative rendering backends. Moreover, Luke Gorrie (of SLIME and Distel fame) is working on an Emacs McCLIM backend: https://mobile.twitter.com/lukego/status/1387747747864973312

The surprise for me, working on the Emacs backend for McCLIM, is that CLIM is actually a lot simpler than it seems.

The CLIM specification is overwhelming. The Lispworks and Franz user manuals for CLIM are too. There are just too many concepts being presented at the same time and with incredibly complex class hierarchies (e.g. a Stream is a Medium is a Pane is a ... I can't keep track.)

But behind all of that there are a few simple and mostly separate frameworks: describing colored geometric shapes and text; associating drawn shapes with application objects; defining commands to operate on application objects.

You can actually pick and choose from these APIs and map them onto whatever "backend" you like. The Emacs backend, for example, initially just translated the geometric drawing commands into SVG elements. That was really easy. Then we added unique IDs to those elements so that we can enable/disable them as input sources - also fairly easy.

The Emacs backend is becoming fairly extensive but it's still only 650 lines of code: https://github.com/nuddyco/McCLIM/blob/clime/Backends/Emacs/...

Alternative backends are old thing, Daniel mainly documented how to make them (other than various cleanups underneath).

The reality is that nobody cared enough about those alternative backends to keep them alive (other than CLX, for a time beagle was usable - but only on OSX, unless someone did a lot of work to compile it for other unices using GNUstep/OPENSTEP)

Well yeah, if there's no documentation on how to create a backend, the barrier to entry is that much higher. It is great that this is now getting addressed.

And none of the old alternative backends, including Beagle, ever worked for me. I certainly tried. I think it's fair to say that all of them were in various states of disrepair and none ever reached the point of "fully working".

I'm pretty sure beagle worked for a time, but that time was probably "CCL/RMCL on PPC mac with OSX" and I don't think it would cover all versions of that.

> Luke Gorrie is working on an Emacs McCLIM backend

This works quite well now. If anyone wants to try it the installation instructions are at https://gist.github.com/lukego/0b74b94492066ae2b8c2a12b18e84....

Yeah I worry about the reliance on CLIM too. I want to like it but it seems so complicated and general despite also not seeming that useful or relevant today. Perhaps it is something that really requires everything to already be in CLIM, or perhaps it is just incomplete and insufficient at the moment due to a lack of resources.

Supporting multiple backends sounds like a good idea but if CLIM is one backend among “many” then it cannot be taken advantage of. But maybe CLOS will allow CLIM to still be exploited.

The backend are implemented “behind” CLIM itself: McClim is designed to abstract over multiple rendering backends. For a while the X backend was the only one that was ready, but I think there’s ongoing projects for a Windows and a Gtk/Cairo backend right now.

This is the kind of backend I was referring to:

> Climacs depends on McCLIM for its graphic user interface. Second Climacs is independent of any particular library for making graphic user interfaces, allowing it to be configured with different such libraries. Though, at the moment, the only graphic user interface that exists uses McCLIM

Hm, Luke has also been working on a GTK backend for McCLIM. I wonder if he gave up on that, or just working on it in the background, or simply put it on pause for a bit?

That must be someone else. The Emacs backend is the first one I'm working on.

So it seems. It's actually lokedhs. I must have mixed up "loke" with "luke".

FYI I use waypipe with the feature/pgtk branch as part of my daily workflow, and it works great---better than X11 forwarding in fact.

> and GTK is required only on Linux

You can build GNU Emacs for X with Lucid instead. Motif too, but that is getting removed soon.

Thanks for correcting.

Some cross-platform text editors like Textadept require GTK on Mac and Windows, but Gnu Emacs does not, is what I meant to say.

For those of you not in the lisp sphere: Robert is a very competent guy. He is probably the guy you would want for this kind of thing.

In school a couple years back I spent fair amount of extracurricular time playing around with various lisps and frequenting related irc channels.

Robert often went out of his way to answer my dumb questions and really helped me better understand a number of things.

Thanks Robert!

Hemlock is another Emacs implemented in Common Lisp. I don't know how it differs from Climacs except the latter obviously uses CLIM as its front end. Modified versions of Hemlock are used as the IDE editors for CCL and Lispworks.

https://en.wikipedia.org/wiki/Hemlock_(text_editor)

It is nice being able to use parts of (or all) of the LispWorks editor in applications.

Yes but...LispWorks doesn't include [most of] the source of their editor, so I cannot modify or customize it much. CCL includes the entire source, which means one can customize it as much as one wishes.

OTOH, CCL doesn't have CAPI.

Hemlock is still delivered with CMU Common Lisp if people want to try it out.

As I've been learning more Emacs Lisp and Common Lisp, I've found that they're really quite similar. So I've been wondering why Emacs wasn't written in Common Lisp. Of course, I think the answer to that one is the age of Emacs -- it was around before CL? -- but this project will be one that I check in on periodically, for sure.

This is the story [1], told by Stallman. But yes, basically because Common Lisp was too big. That is also the reason GNU uses Scheme as a language for extensions.

[1] http://www.gnu.org/gnu/rms-lisp.html

Common Lisp was considered too big in its day but compared to many modern languages it is quite small (only 25 or so special forms, most of which are rarely used; few built in types).

I think the problem was that there was still too much of a separation between the language itself and user written functions. Because so much of the language cares about object identity, one couldn’t write a portable hash table oneself. Making a data structure like a search tree becomes hard because there isn’t a good way to sort arbitrary objects of arbitrary type or embed a specific sorting function and type check. CLOS is both massive and poorly integrated. The language is full of features that are old (rplca), hard to compile (&rest, adjustable arrays with fill pointers whose data is from another array at an offset), inextensible (loop, +), half-baked (types), or difficult (pathnames). But it also misses many features (eg threads). Without a refreshed standard it is hard for the language to move forward while code remains portable.

In some ways I think of Julia and clojure as successors. Julia has types and multimethods working very well together but drops method combinations, pervasive mutability, lists, symbols, and syntax for good macros. Clojure keeps the macro-happy syntax and symbols, bins the mutability, and makes the data structures more compatible, functional and efficient.

Obviously this doesn’t have so much relevance to emacs past or present.

I'd be curious about a Racket implementation. I know Racket is supposed to be a research and teaching language, but in several respects it seems to be a better Common Lisp than Common Lisp itself.

That might have become more true recently because of the Racket-on-Chez compiler, but until then Racket was in most cases slower than Common Lisp because it wasn't AOT compiled to the metal, which Common Lisp is.

I also like CLOS (the Common Lisp object system) better than the object systems available in Racket, as well as Common Lisp's macro system. But that's just personal preference. The existence of Typed Racket is pushing me to explore Racket more.

> it wasn't AOT compiled to the metal, which Common Lisp is.

It's required that a CL implementation supports compilation, but compiling to machine code isn't required. e.g., Clisp has a bytecode compiler.

- https://en.wikipedia.org/wiki/Common_Lisp#Compiler_and_inter...

- https://clisp.sourceforge.io/impnotes.html#bytecode

Though they experimented with a native code JIT...

Here are some lispers' impressions: https://gist.github.com/vindarel/c1ef5e043773921e3b11d8f4fe1...

Racket seems to miss at least two big points: the interactivity/the debugger and the condition system.

Yeah, the Racket/Scheme mindset is generally very different from the mindset of most CL users I’ve come across: despite the superficial similarities, scheme users seem to be less interested in interactive “repl-driven” workflows than most CLers

That would be nice, for sure. At least the Racket IDE editor supports some Emacs key bindings.

emacs in various forms definitely predates CL; but I suspect around the time CL was ratified (84?) any discussions about a re implementation would have focused on the size of CL implementations being unacceptable for inclusion in an editor...

subsets of Common Lisp existed early on

Sure, and lots of lisps existed in the 15 years or so before CL.

The ones that looked anything like CL were pretty heavy. I imagine the decision to write elisp at all in the first place was made by people pretty familiar with what existed at the time, and why it wasn't a good fit for them....

xlisp for example started small in 1983 and over time added some CL features. It was also used by Autodesk as the base of Autolisp for Autocad on a zillion PCs...

> The ones that looked anything like CL were pretty heavy

yeah, Coral Lisp needed 1 MB RAM on a 68k Mac. Morphed into a Common Lisp then.

mulisp in 85 supported 0.5MB RAM on a PC. It had some CL compatibility.

Franz Lisp could probably have run MacLisp Emacs.

Common Lisp didn't exist in a useful state at that time. Furthermore was probably too large for this application at that time.

Also a bit controversial.

I’ve often wished for a rewrite of Emacs—the rendering engine sure could use some love. The hardest thing in my mind is getting compatibility with all the existing Emacs packages. That’s where I derive most of my value from. If it can’t support org-mode, magit, mu4e, racket-mode, etc., there’s no way that I could switch.

The cluffer documentation is really good... But it shows that there's a lot of things still to do regarding performance (for example to edit very long lines). I'd be happy to read something about a proper, performant implementation of a text editor buffer. That seems like a very interesting topic.

The documentation for xi[0] is quite interesting, and the yi parsing paper[1] is also worth a look.

0. https://xi-editor.io/docs/rope_science_00.html

1. https://publications.lib.chalmers.se/records/fulltext/local_...

Robert presented some of his ideas few years ago at one of the European Lisp Symposiums, unfortunately I don't recall the title right now. But should be findable using the ELS site[1]

[1] https://www.european-lisp-symposium.org/

How is this compared to Emacs? Are all of the extensions cross-compatible? I would love to use Common Lisp for everything.

You can use CL for everything[0] :] As for text editors, there's Lem and LispWorks

https://github.com/CodyReichert/awesome-cl

https://github.com/lem-project/lem/

https://lisp-journey.gitlab.io/blog/discovering-the-lispwork...

From TFReadme:

>At the moment, all you can do is type some text, and you can use C-x i to insert and existing file. Some basic Emacs commands also work, like C-f, C-b, C-p, C-n, M-<, M->, and C-x C-c. The visible window does not automatically follow the cursor yet.

So if Second Climacs were to get an Emacs Lisp interpreter, how far along would it be?

> It is called Second Climacs because it is a complete rewrite of the Climacs text editor

Oh OK. I thought the name is a naughty pun. Now thanks to this totally innocent explanation, I know it is :)

"Second Climax" is a well-established literary term for the climax of the second act of a play: https://en.m.wikipedia.org/wiki/Three-act_structure

What's the license?

Second Climacs describes itself relying on McCLIM, which is enough to infect it with GPL.

McCLIM is licensed under lgpl-2.1+ and does not impose its license on applications.

That said using the word 'infect' seems to imply that copyleft licenses are a disease; this does not sound like a serious discourse argument.

It's fairly anodyne to describe GPL3 and friends as "viral" licenses, is it not?

https://en.wikipedia.org/wiki/Viral_license

Stallman doesn't like it, but RMS is renowned for the scope and breadth of things he doesn't like.

To say that something "viral" can "infect" something else is the natural extension of the metaphor. I can see not liking it, but it doesn't scan to me as unserious or in bad faith to say this.

Edit: this kind of language goes both ways between copyleft and permissive camps in FOSS circles. I see objections to permissive licenses phrased in terms such as "improvements to MIT code can become locked away inside proprietary software", which only works if you consider proprietary software to be a prison, when this is in fact an intended outcome of permissive licensing.

I thought the 'infect' was just used because it's a 'viral' license - it fits the medical analogy/broad usage of the terms 'viral' and 'infect'.

Is there a standard way to put it that doesn't have negative connotations?

Infect implies it's involuntary. Nothing's involuntary in a new project like this. The standard way to put it is "so we went with GPL" or even just "it's GPL".

I affirm that I misrepresented McCLIM's license despite looking directly at it. I appreciate that you corrected my error.

Depending on how modularly Second Climacs combines McCLIM into itself, it is possibly LGPL licensed or, at minimum, released under the Github terms of service: view and fork. I am not in a position to assess the level of integration Strandh has opted for.

The Gnu Public licenses are designed to proscribe the minimal license features required for works combined with those released under GP licenses.

No, my assessment is not a serious argument, it's an incorrect puzzle solution, during lunch.

Only when you distribute Climacs with McCLIM included. If you distribute it with another backend, the code could be licensed with any licence.

In any way, they should attach a license.

Not even a screenshot? :(

Reply on news.ycombinator.com