Show HN: Parinfer – a simpler way to write Lisp
shaunlebron.github.io
shaunlebron.github.io
Paredit always ended up being a requirement for me, but this takes it to the next level. What other editing problems out there could have excessive hotkey usage replaced by something automatic with the same level of power, I wonder? This is a great example of an area where user interaction experts can really have a lot of impact, and I think shows why engineers ought to be more accepting of disciplines like that.
Have an early prototype of Parinfer working with the Atom editor here: https://github.com/oakmac/atom-parinfer
It's pretty early software, but feedback appreciated :)
I'm glad it "just works"!
And I already wanted to learn Lisp. Because I kind of half-assumed this tool to be there already :-P
I always assumed, obviously Lisp didn't get this popular if people had to keep track of all their parentheses. Sure they probably had to in "the old days" (although I wonder even when?). It just seems like a tedious thing to have to do in order to work with such a consistently structured language.
Of course you can get most of it with highlighting matching parens and auto-formatting shortcuts. Like most languages.
But I kind of expected Lisp tooling to go the extra mile. Because it is that structured.
From my limited knowledge of Lisp, I figured some vague ideas on how such a tool would work / should be designed. It's kind of cool to see that the author's initial steps match up with that idea: define a grammar, some invariants, and "have at it" (ok, so I didn't figure that far). And then he made it work!
Maybe a silly question, but since the author is in this thread: Was it hard to make? :-)
BTW the idea of switching between these two modes is a very clever approach to keeping certain invariants invariant while still being able to modify the code--because the trivial way to preserve the invariant would be the tool immediately reverting your edit :)
You could borrow the idea of indentation inferring function scope structure and make a plugin that would modify JavaScript syntax accordingly, but I'm not sure that would even be desirable when writing JavaScript.
Parinfer is made possible by Lisp's unique syntax.
http://emacsrocks.com/e14.html
I would deeply love to see this kind of inference engine ported to emacs as an alternative to Paredit's chords
This makes it even better:
(defun my/lisp-dedent-adjust-parens ()
(interactive)
(save-excursion
(x4-smarter-beginning-of-line)
(call-interactively 'lisp-dedent-adjust-parens)))
(defun my/lisp-indent-adjust-parens ()
(interactive)
(save-excursion
(x4-smarter-beginning-of-line)
(call-interactively 'lisp-indent-adjust-parens)))
(local-set-key (kbd "<M-left>") 'my/lisp-dedent-adjust-parens)
(local-set-key (kbd "<M-right>") 'my/lisp-indent-adjust-parens)
Oh, not again.
The author is confusing s-expression syntax with Lisp syntax. Even s-expressions are more than lists (conses, cyclic conses, symbols, vectors, strings, various numbers, structures, ...) and the s-expression reader knows about more than pure s-expressions - for example comments.
Not every s-expression is a valid Lisp expression. For example (let "let") is not a valid Common Lisp expression. The syntax for LET is:
let ({var | (var [init-form])}*) declaration* form* => result*
Lisp has built-in syntax (provided by special operators in Common Lisp) and arbitrary complex syntax provided by macros.
This is valid for other languages with a notation based on s-expressions, like Scheme or Clojure.
That sentence is an oversimplification to give context to the rest of the content in that section. Should probably have an asterisk to clarify.
Everyone who reads or hears this sentence. I've heard 'everything is a list' a zillion times already. It's just wrong. In every way.
A Lisp program is data. The Lisp system does not care if it comes from a text file or if it is constructed as data by a program:
(list '+ '1 '2) constructs a program which can be evaluated."code n. 1. Trad. any representation of actions to be performed, whether conceptual or as an actual object, such as forms, lambda expressions, objects of type function, text in a source file, or instruction sequences in a compiled file. This is a generic term; the specific nature of the representation depends on its context. "
http://www.lispworks.com/documentation/HyperSpec/Body/26_glo...
So, it's worth mentioning and replacing with something more accurate.
(john "123" 1) consists of a list, a symbol, a string, an integer
|hi there| is a symbol
#*1010101011 is a bit vector
#2A((1 1 1) (1 1 1)) is a 2d array
Also look up the current Scheme report and see its syntax definition. 'Everything is a list' is fooling the reader and treating him/her like an idiot.
If you pick up just about any book on Lisp, it's all about recursion and lists. I don't remember SICP talking about native vectors or hash tables. I don't remember Touretzky talking about those either (maybe it was just that I had an early edition (1988-ish?) or my memory was faulty). Certainly The Little Lisper/Schemer never talks about them.
The first book I remember talking about other primitives in Lisp was Siebel's "Practical Common Lisp".
There are quite a few pernicious fallacies the swim around the "everything is a list", "S-exprs are easy to parse because they're uniform" (oh, really? Try identifying a list in the presence of dotted pairs), etc.
But surely after like 10 minutes of doing anything with it?
I haven't even begun to learn Lisp, just know that it exists and can mostly intuit/read it when digesting articles on various CS-related topics. It's pretty clear that there's something more than that going on.
Otherwise a whole lot of things that make Lisp interesting don't make a lot of sense. Where does the meta-programming come from if it's all just lists? How can it not be horribly inefficient if it's really just lists? What's so great about lists if really all you have is just lists, anyway?
So I'm going to go ahead and claim that most people that are intrigued enough to want to learn Lisp, already know (or assume) this. Seems like a great hook to start, actually: What is it about Lisp that's so great if it's just made of lists?
Now from what I read in your comment, you got Lisp dumped on you in CS college? "Here's this thing you have to learn about" is a fine way to learn about stuff (so many super-useful topics I probably would never have touched if they hadn't been forced on me in CS college[0]). But without a personal motivation of "I read everywhere that this thing is so powerful, and I'm going to find out why", then indeed I can imagine how "everything is a list" can be a toxic thing to teach about Lisp, in particular to a class of students who are busy enough that they just want to pass the course. What if they forget most, except that bit?
[0] except for me, Lisp wasn't one of those. We got Haskell though. Except it came at a rather inopportune moment, switching curricula, had to tag along last moment with some friends for the assignments, crammed for the exam, somehow passed and promptly forgot what monads are again. boo, hiss. yes. And I still managed to learn a lot from it! :)
I did often wonder where the arrays, hashmaps and other data structures were. I remember I had my own practice problems that involved random access so a vector would have been more efficient than a list.
In fairness, I think you either will be far enough on your travels with other languages to intuit that Lisp implementations do have these primitives ("surely not? Ah, lo and behold, racket has a nice vector interface. Not sure about CL's hashmap interface..." and so on) or you will be so far on your travels with lisp to understand why you need this that you already know.
I don't think I would direct a complete beginner to programmer to this page.
> One kind of primitive expression you might type is a number. (More precisely,
> the expression that you type consists of the numerals that represent the number
> in base 10.) If you present Lisp with a number
>
> 486
>
> the interpreter will respond by printing5
>
> 486In Lisp the cons structure is actually the primitive. Lists are built on top.
Indeed the article begs for potshots from smug lisp weenies:
1) It's clearly talking about Clojure, yet leads with "Let's simplify the way we write Lisp". Them's fighting words in the Lisp corral.
2) Paredit clone, but only acknowledges that for animations.
3) No emacs version! (Understandable given #2 however)
Then again what he says is true for 99% of the cases, and a good simplification.
>Even s-expressions are more than lists (conses, cyclic conses, symbols, vectors, strings, various numbers, structures, ...)
Still expressed as lists when it comes to the syntax level.
It's not. Read it slowly: only a subset of s-expressions are valid Lisp programs. Examples for invalid Lisp programs, even though valid s-expressions:
(cl:if x > 1 t nil) ; IF takes 2 or three forms
((cl:lambda) (a) a) ; not a valid lambda expression
(cl:let "a let expression" ((a 1)) (+ a 1)) ; the second element should be a list of bindings
(cl:defun foo args (length args)) ; arguments needs to be a list
(cl:let ((a nil t)) a) ; a binding is a variable or a list of symbol/value
Lisp syntax is not the syntax of s-expressions. Lisp syntax is defined ON TOP of s-expressions.
> Still expressed as lists when it comes to the syntax level.
Comments are not expressed as lists. Feature expressions are not lists. etc etc...
Sure, but the thing we're debating is whether "all s-expressions are valid Lisp programs" (that would be a miracle actually), but whether Lisp syntax is all lists.
And all those examples are still lists -- that the lists can also contain other items inside them, or that underneath they are implemented as cons etc, doesn't really change that.
>Comments are not expressed as lists.
They are not retained anyway when the strings are parsed to the "live" AST (AFAIK), so they are an exception to the other syntax in that too.
> Most programming languages have several syntax rules. Lisp has one: everything is a list. The first element is a function name, and the rest are its arguments. Thus, the language is simply a collection of compile- and run-time functions, trivially extensible.
Step by step.
> Lisp has one syntax rule: everything is a list.
That's wrong. completely wrong. 'everything is a list' is not THE ONE syntax rule of Lisp. Lisp, like most other programming languages, has MANY syntax rules.
A defining feature of current Lisp dialects is even that the syntax rules are programmer extensible and thus Lisp has an unbounded number of syntax rules. Thus the complete opposite is true: Lisp has arbitrary many syntax rules.
> The first element is a function name, and the rest are its arguments.
That's also wrong. It always was. Lisp has since day zero supported more than just function call forms.
Valid Lisp expressions:
3
"1 2"
((lambda (a) a) 1) ; (lambda (a) a) is not a function name
(defun foo () ()) ; defun is not a function name
(let ((a 1)) a) ; let is not a function name
I would not even know what that actually might mean...
Lisp like most other programming languages is not defined by one syntax role.
One of the main features of Lisp is actually that it supports user-defined syntax - either on top of s-expressions (aka MACROS) and/or by changing/extending the syntax of s-expressions (aka READER MACROS).
Wouldn't you want to know that? Just let the user of this editor feature know that it mainly works on the list level and does not know of the full syntax of Lisp and that this is sufficient for a wide range of editing tasks.
The emacswiki for Paredit says:
http://www.emacswiki.org/emacs/ParEdit
> ParEdit (paredit.el) is a minor mode for performing structured editing of S-expression data. The typical example of this would be Lisp or Scheme source code.
> ParEdit helps keep parentheses balanced and adds many keys for moving S-expressions and moving around in S-expressions. Its behavior can be jarring for those who may want transient periods of unbalanced parentheses, such as when typing parentheses directly or commenting out code line by line.
It says what it does. It makes no claims about Lisp syntax.
Let's try to not oversell Lisp syntax - Lisp syntax has been controversial for decades and legions of students and programmers struggled with it. There is a reason for that.
Tell a student that Lisp has only one syntax rule, the list, and then he/she struggles a lot forming valid Lisp programs?
Typical questions:
(if (> sin (a) pi) 'foo 'bar)
(cond (> (sin a) pi) 'foo
(t 'bar))
Let's be honest, it's not the students fault. The syntax of Lisp is actually more difficult than just writing lists.
I don't think this is 'pedantry'. Teach Lisp to people or deal with programmer's typical problems and you'll see lots of these questions.
alists are not expressed as lists.
Very nice.
I just want to download stuff from my head to the editor before I forget it, then later fix the mess, and have the IDE/Editor still help me in the meantime. If it needs to temporarily revert to a caveman-style indent-like-the-previous-line or event to temporarily disable auto-indent, I'm ok with that, but don't horribly mess up what I type in failed attempts to semantically auto-indent broken code that I don't wanna fix right now...
Let's hope they do, soon!
[0] 35 years, in my case. David Chapman and I implemented balanced-parenthesis editing commands for Zmacs (the Lisp Machine editor) in 1980. To my knowledge, it was the first such implementation for a text editor; the Interlisp structure editor already existed, but was considered hard to use.
https://meaningness.wordpress.com/about-david-chapman/
I'm something of a fan. This article is great, about his history at MIT's artificial intelligence lab, Ken Wilber, Heidegger, and Buddhism.
http://meaningness.com/metablog/ken-wilber-boomeritis-artifi...
While sleuthing a bit on Google I realize he was also a contributor to the "UNIX-HATERS handbook." Weirdly, that was really fascinating literature to me as a ... 15 year old, maybe, in like 2004? In my Swedish teenage room full of messed up Linux and OpenBSD boxes, reading everything I could that seemed interesting and intelligent. That handbook, the C2 wiki, various mailing lists, Usenet groups, IRC channels... I attribute a lot of my personality and interests to this nerdy/literate heritage.
Maybe others have different opinions, but I find this way of doing things a lot more natural and usable. It's like an old Oblivion mod for archery that asked the question: "Why do other archery mods have a hotkey for denocking your arrow? We just automatically denock if you look at the ground. It just makes sense."
Trying it out in Atom, it looks really promising. You just kinda start writing, like on the blank line at the end of a function body, and Parinfer fills in the parens depending on your indentation level in an intuitive fashion, merging your form into the tree above it.
In fact, it seems to strictly complement Paredit. Parinfer puts the parens where you want them, and Paredit is there for explicitly editing the tree.
Some ways that Sublime is more familiar are that it uses the OS-standard editing hotkeys, such as moving by words with Ctrl-arrows or Option-arrows. It also uses OS-standard terminology, like “close window” instead of “kill buffer” and “paste” instead of “yank”. The result is that new users don’t have to learn anything to be productive – they just edit as if they are in Microsoft Word, while enjoying the programming-specific features of a text editor such as syntax highlighting and block folding.
The better defaults include easily creating more than one scratch buffer (just run “New File” multiple times), automatically balancing parens when you type, support for pixel-by-pixel smooth scrolling, and scrollbars that don’t confusingly shrink when you start scrolling past the end of the buffer. I’m sure Emacs supports all these things, but you have to manually configure it to do so first. Many users are scared off from Emacs after seeing that it requires up-front investment to get features that other editors already provide with no work.
To whom? There is still a huge number of incompatible text editor cultures. Mac bindings, Microsoft bindings, Wordstar, etc.
> better defaults.
There are dozens of "better" .emacs pre-cooked packages. Choose any. No need to stick to the out of the box defaults.
> OS-standard
Which OS?
> he result is that new users don’t have to learn anything to be productive
The "new" users had to learn all those "OS default" bindings and terminology first. Not that "new" in my book then.
> it requires up-front investment
They simply forgot their up-front investment into learning the defaults of their chosen OS. I understand that - I still cannot use Microsoft Office, can't be bothered to learn it.
So, the only advantage is familiarity to those who already invested a lot of time into learning something else. Nothing of a value for the new users.
Apart from defaults, I agree that GUI text editors provide no advantage to completely new users. But I’m not sure that “new users” of text editors, unfamiliar with OS conventions, is a category worth talking about – I think it’s really small. It shouldn’t take more than a year for a new computer user to absorb the OS conventions while browsing the web and writing documents. And how many people might want to try out Emacs within their first year of using a computer?
I guess that about 10% of eventual programmers started trying out programming in their first year of computer use – the other programmers started later in life. The number of fledgling programmers who care about editors enough to look into Emacs has to be even less than that – it’s probably something like 1% of all programmers. The other 99% of programmers will find GUI editors more familiar than Emacs, because they have already internalized the conventions.
Emacs is a GUI text editor too, btw.
I can be wrong, but my impression was the opposite - all the new tutorials I saw recommended one or another starter pack with all the bells and whistles out of the box.
> while browsing the web and writing documents
I'm not sure if a proportion of users who write documents in a specialised software (like Word) is significant enough. And web browsing is largely clicking, no hot keys are used in general, so no habits to be picked up.
Another thing to consider is that now more and more users are coming from mobile platforms, without any desktop exposure whatsoever. For them, any desktop conventions are equally alien.
> I'm not sure if a proportion of users who write documents in a specialised software (like Word) is significant enough.
I thought that most computer users learn to use computers while in school, for which they use a word processor to write essays and reports for the teachers. From around 2004 to my graduation in 2009, my middle and high school’s teachers even required essays to be typed and printed, so that they wouldn’t have to struggle to read students’ handwriting.
I guess some students prefer writing essays on paper, but those students probably wouldn’t become potential Emacs users.
Am I missing a common backstory for programmers that doesn’t involve using computers to write reports for school?
And with all the tablets and smartphones, even the basic touch typing skill is now extremely rare among the young.
Having to indent stuff manually is a drag. Especially when you nest in deep with Python and need to close several levels of indentation. It's just inherently difficult to "measure" white-space with your eyes. With 2-space indentation which is used in Lisp, and naturally deeper nesting, it's probably even harder.
Sublime is trash compared to Emacs.
Here's one such experimental editor: https://github.com/darwin/plastic
Edit: I see now that Parinfer refers to Plastic under its Acknowledgements section. :)
Had I known how polished my workflow would be a year later with Paredit + Emacs in-buffer line-by-line code evaluation, or that tool-supported s-expression editing would let me write/refactor/move/nest code much faster than in other dynamically-typed languages, I would've stopped what I was doing and started learning Clojure immediately.
But you don't know any of that if you're someone going to http://www.tryclj.com for the first time.
Seems like Parinfer could be used in environments where you don't have Paredit available or, perhaps more importantly, where the user hasn't already credentialized in a niche tool for editing s-expressions.
What I can't understand is why 'C Style indentation' isn't the default convention, at least in modern user-friendly Lisps. One of the biggest problems people have with Lisp is the readability of grouped parentheses, but the C Style indentation example shows you can write Lisp without them.
I get that it's nice when programs fit in less lines of code, but isn't readability more important than line count?
The purpose of C brace style is to make editing easier in primitive editors that don't do delimiter matching. In such an editor, selecting where, in a long string of closing delimiters, to insert a new element requires counting the delimiters, which is tedious and error-prone. With delimiter matching, the editor does the work for you, so splitting delimiters across lines is completely unnecessary.
When execution control depends on the order of nested arguments, I find "c" style to really enhance the following contrived example, even with rainbow parens:
(if true
(whatever this is an (example
that is (kind of hard)
(-> to
(read %))))
(this is an ELSE but hard to figure out)) if (x
> 100) { console
.log('hello') } else
{ console.log
('world')}
Meanwhile, I can't even write your lisp example as-is in any of the editors I tried it in.
Also: http://imgur.com/FnXXOXJ
Tell me the above picture is something people have problems understanding the structure of.
Also you have your indentation wrong; (this is an ELSE but hard to figur eout) should be one column to the left, which makes it significantly more readable.
To this day I have not found a tool that works as well as Paredit for editing and navigating s-expressions.
Parinfer looks like a cool subset of Paredit's abilities. I'd love to see it expanded to include all of what Paredit can do.
EDIT: I was mistaken, Parinfer does things Paredit doesn't. They could totally work together, that'd be pretty awesome.
I'm more concerned that Parinfer appears to be modal, based on whether you manage indentation or parens. This seems overly complicated; it would be nice if there's some elegant way to merge the features of both into one mode. But I don't know how to do this, and I haven't actually used Parinfer, so maybe there's nothing to worry about.
https://github.com/Fuco1/smartparens
Basically paredit on steroids.
The readme is not "a lot" in the slightest, the first paragraph of is a perfect summary. I'm sorry for being rude, but if you can't even bother to read one measly paragraph, I'm probably wasting my time writing this comment for you. The first paragraph is barely even longer than your original comment (10 words longer)!
The readme also links to a list of features and an article with gifs. Additionally, I gave you an even shorter and simpler tl;dr in my original comment: "basically paredit on steroids".
https://github.com/Fuco1/smartparens/wiki/Paredit-and-smartp...
I think I might try it out. It seems by default it's less strict than paredit, though, so I'll have to tweak it a bit.
Yes, but there is a strict mode you can turn on. Personally, I use the default with a couple binds that make it a bit more strict, which actually make it very pleasant to use, such as:
(global-set-key (kbd "C-<backspace>") 'sp-backward-kill-word)
(global-set-key (kbd "M-d") 'sp-kill-word)
(local-set-key (kbd "<backspace>") 'sp-backward-delete-char)
(local-set-key (kbd "C-d") 'sp-delete-char)Code, for me, is a means of communication with both the machine and the human reader. The machine doesn't care about lisp indentation but code format matters when trying to communicate with human readers. Communication with future readers, including yourself, is what makes programs maintainable.
Pretty-printing is fine since the code format follows a standard, which makes it easy to ignore and easy to read, generally a "good thing". On the other hand, there is information in the choice of code format, similar to the information found in a poetry format.
Properly indented code is "good code" and, over time, is likely to be what flows from your fingers regardless of the editor. But excellent code is an art form that requires thought. Lisp is a language where indentation is meaningless but shape communicates. It is not something to delegate to an editor.
> And alternative syntaxes (Lisps without parens) have faultered since they sacrifice some of Lisp's power that seasoned users aren't willing to part with.
Could Elixer be an example of one of these Lisps without parens? [1] Can anyone give an example of what kinds of Lisp powers are sacrificed in a syntax where the parens are implicit rather than explicit? I can't think of anything off the top of my head.
It seems to me that Parinfer, when working as intended, almost turns a regular Lisp into one of these parenless Lisps, by performing parens manipulation for us, which in turn allows us to treat whatever Lisp we're working with as if it was a parenless Lisp.
[1]: https://blog.8thlight.com/patrick-gombert/2013/11/26/lispy-e...
define factorial(n)
if {n <= 1}
1
* n factorial(- n 1)
(define (factorial n)
(if (<= n 1)
1
(* n (factorial (- n 1)))))
But the author seems to have overlooked that sweet-expressions, which were developed later, do not sacrifice homoiconicity. A human reader can still easily understand the AST from reading sweet-expression syntax. The real reasons that sweet-expressions have faltered relate more to the increased difficulty of collaborating with other developers, the lack of editor plugins that highlight sweet-expressions correctly, and incompatibility with Clojure syntax.
Personally I don't care to write in lisp since I loose track of parens way too readily. This project might be very handy!
In the meantime I've come to view Julia as my goto lisp. Quite fun to be able to write hygienic macros, but still have a mostly familiar imperative syntax for common tasks. Actually, the disciplined (read limited) usage of macros in Julia is a big draw for me, since I've heard too many horror stories of macro readers and spaghetti code in large lisp projects. Interesting look at elixir, I'll have to play with it sometime.
The rules for what happens when inserting/deleting parens must be
learned. Also, the case necessitating a "Paren Mode" comes at the
cost of forcing the user to understand when and how to switch
editing modes.
Also, the preprocessor step performed when opening files will cause
more formatting-related changes in your commit history when
collaborating with others not using Parinfer. they have to make the indentation part of the syntax
shifts cognitive load from managing parenthesis to managing indentation
(foo [1 2 3
4 5 6
7 8 9])
(foo [1 2 3]
[4 5 6]
[7 8 9])
edit: Ah, it works if you put all the forms on one line. I suppose it would be considered non-standard to indent that particular syntax in such a way.
Parinfer will infer the closing brackets.
(foo [1 2 3
[4 5 6]
[7 8 9]])I haven't been doing much Clojure or Common Lisp development this year but I am putting Parinfer on my look-at list.
Even if the code at rest is in S-expression syntax, editing it in this syntax would be slick.
There can potentially be a "big diff" when you run Paren Mode on a file for the first time, but that's typically a one-time event. The beauty of Parinfer is that most Lisp code already uses these conventions.
Once you start using Parinfer's Indent Mode regularly it feels crazy to write or edit lisp without it.
One of my plans for atom-parinfer[0] is to have a comment flag in a file that will automatically signal "use Parinfer on this file". Similar to JSHint inline configurations [1].
[0] https://github.com/oakmac/atom-parinfer [1] http://jshint.com/docs/#inline-configuration