Ki Editor - an editor that operates on the AST
ki-editor.org
ki-editor.org
Ctrl + W
Ctrl + Shift + W
It really changed my perspective on interacting with the 'text' of a file.
VS Code, Zed, etc. have similar operations, but in my experience they expand and shrink too coarsely.
I love AST aware editing. I think it's one reason it's always been so nice to edit lisps. Stuff that is complicated to describe in javascript (and doesn't have LSP support) p much requires a whole AST parser, but in lisp it's just a simple list operation. When I go back typescript after a weekend of clojure, I reeaally miss slurp! and other paredit commands
The shortcut I use the most in Jetbrains IDEs. Also the one I miss the most in VSCode (whatever is present there just doesn't seem to work right).
Also the shortcut that has caused me to close so many browser tabs inadvertently...
Cmd+Shift+V - Stacked clipboard, you can start typing to search or hit a number to choose what to paste (keeps everything you've copied/cut inside jetbrains for a while)
Cmd+Shift+E - Recent locations, you can start typing to search - shows little buffers of where you've been recently
Cmd+Shift+A - Action tab of the command palette - fuzzy search for any command (really the only shortcut you need, other than maybe Shift+Shift for main command palette shortcut)
--- Through the Action bar...
Local History / Local History of Selection - you can start typing to search quite far back the history of all changes of the current file or selection - you can also right click a folder or the project and do the same. Much finer grained than git.
The general concept of being able to search for something and edit directly in the buffer of the search results.
I will add one that are possibly more well-known:
- ctrl + shift + F: Find text in any file
- ctrl + N: Find types (structs, classes etc)
- ctrl + shift + N: Find any file by name or pathAlthough i do use cmd+shift+r for global replace
The wonderful Flycut (https://apps.apple.com/us/app/flycut-clipboard-manager/id442...) fixes this on macOS.
[1]: https://github.com/gritzko/librdx/tree/master/be#readme
An overview video: https://www.youtube.com/watch?v=XGm_khXZl44
I tried it, but it just was too clumsy. Sometimes refactoring/editing needs to go through phases where the AST is invalid, and MPS makes that just too clumsy.
But with AI this might be a different story.
The challenge is getting this to be a useable way of entering programs. I think I made progress on this, but the feasibility varies with the programming language.
I can't run it any more, since the display hardware it assumed is no longer available, but you can read about it at https://ucalgary.scholaris.ca/items/da8b823b-c344-4ffb-aa37-...
Well exactly.
When the path between Program A and Program B can only be valid programs, you are going to end up with either a much longer, less intuitive path, or deleting everything and starting again. It can also be quite possible to invent structures which are valid but have no valid path to creating them.
And it's certainly possible to create any valid AST in the editor I describe. The set of valid trees is extended to those with "holes" in places, which one fills in when entering a program, and it's always possible to do this.
The challenge is one of finding an intuitive user interface, not whether it's possible at all. One issue is that infix notation is unnatural for entering trees (prefix is more natural).
I'm curious if you have an example of such a structure?
Pedantically: if, for every valid tree, there exists a bidirectional path to the empty root node, there's always at least one path between all given pairs of valid trees ... albeit one that no developer would ever take.
I was remembering a professor I had who was very into formal verification and had an IDE that would only accept valid programs - however his validity checks were more than just tree based, they involved passing a type check. Which now you've called me on it, is quite definitely beyond valid AST editing.
The base case: if you have a structure with a mandated cyclical reference that doesn't accept a second (e.g. nil) type, you cannot construct it without creating an intermediate invalid program. This doesn't tend to show up if the cyclical reference is all the same type (A1 -> A2 -> A1) because you can often cheat and self reference, but it does if it is different types (A1 -> B1 -> A1). You can't construct an A without a B, which cannot be constructed without an A.
But that's still not a problem you cry! And quite right, you can edit the type itself to remove and re-add the reference.
That is until the type is built into the language, or a library.
No, I'm not claiming to have read all one hundred pages already. However, from what I have read I'd love to see a functional demo.
¹ https://simh.trailing-edge.com/
(Maybe your paper is famous and it’s not wild that I read it, but it was wild to see after so many years)
I never took that path, spent time in tech industry confused why people didn’t seem interested in structural editing and better editing tools.
Out of curiosity, how do you think LLMs and genai affect the value of structural editors and similar tooling?
Part of me wants to stay disciplined— of course it’s valuable to work efficiently and work on the AST and with a repl. The other part of me gets paid to work on essentially a punch card system (building dev, ship to prod and see what happens)
The hard part is that we need to be able to talk about these structures. Even just here on this forum we need to be able to communicate precisely about them. I often use · as a typesetting symbol so that I can easily write and read expressions like 2 + · which you would read as "two plus gap". The · symbol is only for typesetting as I say thought because you it's not safe to assume that any one character is reserved for our use in every programming language. Instead we wrap the parts that aren't syntactic in quotes and we use <//> as the symbol for a gap so that it looks more like this:
<*> "2 + " <//> </>
We can parse 2 + · into a proper tree now:
<BinaryExpression>
left: <*Number "2" />
#: " "
operator: <* "+" />
#: " "
right: <//>
</>
It was an experimental language that they wanted to replace their current application level language and was built ontop of JetBrains MPS https://www.jetbrains.com/mps/ which has that feature among others.
My personal opinion after working with that lang is all of this is that its theoretically interesting. But a dead end in practice along with visual coding etc.
The universality, simplicity, and legibility of text as interface for both humans and machines is too hard to beat. I think LLMs is just the most recent example of this.
Things that don't work in practice:
- You need a special editor, its heavy and slow, you lose all the ecosystem around them.
- You can't just cat or inspect the raw file, you can't see it in terminal at all.
- You need a new version control system, review system, and people need to learn it.
- You can't use any existing tools for working with code, you are essentially starting from scratch, and lose all the benefits of all the work everyone else is doing around tooling, dev saas etc.
- humans don't think in trees, they don't think in syntactically correct programs. Its actually absurdly frustrating writing a program in only syntactically correct edits. 99% of the time you are coding, your work is probably syntactically and semantically incorrect.
The tooling that is able to either make the right tradeoff around strictness or allow the users to make the that tradeoff is what ends up being used in practice. A good example of this typescript with gradual typing, python with type annotations, etc.
I think these editors just fall in a bit too far right on the strictness scale.
A more fundamental issue (which I mention in the document) is that there is a discordance between the 2D textual display and the underlying tree. I think this becomes more apparent when input is not only keystrokes, but also mouse positioning.
Of course, these disadvantages do not necessarily outweigh the advantages of editing in terms of trees!
I'm not sure if I can agree with your other points. I can't think of the last time I cat'd or maniuplated rust/java/go outside a heavy and slow editor. Many languages have more or less "one" editor with all the features that most people use, and often use it for change review too. And it seems weird to characterize needing new tools as a "dead end"... all languages need new tools and won't have them at the start.
Plenty of people use go, rust, and java which are not gradually typed.
I was really excited about the concept and read some papers on it at the time. Then I ran a real world application of it, and writing code in it was just a bad experience.
I think I didn't explain the gradual typing properly. I mean the broader concept of moving developers from one way of doing things to another.
When the vast majority of people and tools are working a certain way. Introducing a whole new thing that requires them to work a whole different way is a hard sell.
So having a way for people to move gradually from point A to point B at their own pace and pick up benefits along the way is really nice.
One example of this is Typescript with gradual typing. Swapping form js to typescript is fairly easy. You can still use your old tools and over time you can start to adapt new features into your code, and over time everyone gets used to this new stricter/better approach.
I think ast based code like this essentially requires you to throw out everything all at once, not just editor, but literally everyone's workflow today and rebuild from the ground up.
I just don't think it will ever be more than a niche thing unless it can loosen up on strictness and play nice with existing tooling.
The extent of my usage is having nice textobjects to easily interact with arglists and functions which aren't native to (neo)vim. Very cute and nice to just write "daf" somewhere in a function and just have it "just delete". Or hook it up with basic macros: search for regex, "daf".
I guess it's hard for me to edit things that I don't see right in front of me or aren't super simple changes (like name changes). Or at least, basic things I can reason about (such as finding by regex then deleting by textobject or something).
As for LSP's, I do use go to definition and rename all references, which is nice. But the huge structural refactoring part I have never really done. I don't really use many LSP features besides those two either...
Basically, I gotta up my editor game.
This is actually what's nice about tools like ast-grep. The pattern language reads almost like the code itself so you can see the transformation right in front of you (at least for small-scale cases) and reason about it. TypeScript examples:
# convert guard clauses to optional chaining
ast-grep -pattern '$A && $A.$B' --rewrite '$A?.$B' -lang ts
# convert self-assignment to nullish coalescing assignment
ast-grep -pattern '$X = $X ?? $Y' --rewrite '$X ??= $Y' -l ts
# convert arrow functions to function declarations (need separate patterns for async & for return-type-annotated though)
ast-grep -pattern 'const $NAME = ($$$PARAMS) => { $$$BODY }' --rewrite 'function $NAME($$$PARAMS) { $$$BODY }' -l ts
# convert indexOf checks to .includes()
ast-grep -pattern '$A.indexOf($B) !== -1' --rewrite '$A.includes($B)' -l ts
Sadly coding agents are still pretty bad at writing ast-grep patterns probably due to sparse training data. Hopefully that improves. The tool itself is solid!
If your editor of choice supports an extension (vscode does for example) it's a very easy on-ramp for a better search/replace than regex offers. It's syntax-aware so you don't need care about whitespace, indentation etc. Very easy to dip your toes in where a regex would get complex fast, or require multiple passes.
I converted a codebase from commonjs to esm trivially with a few commands right after first installing it. Super useful.
I hope LLMs eventually start operating at this level rather than raw text. And likewise for them to leverage the language server to take advantage of built in refactorings etc
I don't find myself doing huge structural refactoring often enough that it would be worth my while to learn a specialized tool for doing it; I'm a quick typist, and it's easy enough to just blat through the changes by hand on the rare occasion such a thing is necessary. But I don't work on giant line-of-business apps or sprawling web services, and I can see someone who did a lot of that sort of work having a different take on the matter.
1. Orthodox. Mostly focused on looks and integrations.
2. Modal, Vim improvement. Focus on keeping basic Vim keybindings with minor improvements.
3. Modal, rethinking Vim approach.
Ki falls into the third category which I constantly monitor.
Which is Emacs.
Scripting language? Check!
Custom commands? Check!
Windows management? Check!
Build tools integration and error-based navigation? Check!
File manager? Check!
Looking at the Emacs and NeoVim codebases in GitHub: Emacs is 74.6% ELisp or CommonLisp and NeoVim is 71.9% Lua or VimScript. Never mind, NeoVim is close!
But, can you at runtime modify the NeoVim core functions? Honest question because I've only used vim and VimScript was definately limited compared to elisp. If so, then I say NeoVim and Emacs are both highly-extensible editors.
Emacs is a general purpose programming/computing environment that ships with a text editor and some other nice features. Vim is a text editor and some other nice features that ships with a scripting mechanism. Not the same thing.
(This is spoken with something close to affection; I look fondly on my former Emacs days. I'm probably more likely than not to enjoy the company of a human who has at least tried Emacs and had some thoughtful reaction to it. I may not use it now, but the ideas live on in how I think and what I expect. Editors that push the limits, whatever those limits are, are part of the old school ethos I love about programming.)
So Ed Visual Mode Improved Improved!
> First-class syntactic modification
> Notice the comma between the current and the next node is also deleted. > Notice how comma is added automatically.
This is awesome! And I bet it arguably requires less logic to do so as well. Cool stuff.
Now I'm wondering how much effort it would be to get a ki integration (or at least an AST-first rewrite) in Zed
You can already to a lot in Neovim with plugins, including recreating Helix, Ki, whatever. But the problem is that the vim state machine is always present underneath it, resulting in unwanted behaviors or impossibility to implement some things. I would envision it to pretty much allow to listen on raw keyboard and mouse events. However cursed it sounds, it should be possible to fully implement vscode or any other editing model in neovim
By "recreating everything" I was also referring to the ecosystem around the editor
I think the idea's good, but I don't think you could modify Neovim to support it while retaining the ecosystem – not without also modifying much of the ecosystem, at which point it may be easier to do a straight port to a different editor, or construct a compatibility layer (essentially reimplementing enough of Neovim in the new editor that the ecosystem still works).
nmap can shadow the built-in commands, replacing them with whatever function you want.
> There some cases were Neovim already wants to disconnect itself from vim legacy like allowing to scroll viewport without moving the cursor
I think you mean beyond where the cursor is so it's hidden, but you can jump the screen so the cursor's current position is at the top/middle/bottom without moving the cursor using zt/zz/zb.
This is pretty much how the BRIEF editor for programmers worked, which was used in MS-DOS around 1990. In my opinion this was one of the best programs that I have ever encountered and I have never encountered again some of the good UI features that existed in the BRIEF text editor and in the XTree file manager, both for MS-DOS.
BRIEF had a tiling-window user interface with optional menus and it came with a default behavior, but it could be customized extremely. For each kind of key press or mouse event you could bind a function to be executed, which replaced the default behavior. For instance, the default behavior for pressing most keys was to invoke a function "self_insert", which inserted in the current text buffer and the current cursor position the character corresponding to the pressed key, but it was easy to change the behavior to anything else, e.g. to do auto-completion or inserting some program template.
The functions bound to events were written in a custom language similar to LISP, so in this respect BRIEF resembled Emacs. A great number of text manipulation functions were provided, so it was easy to describe complex actions in the event handling functions.
This was what I liked most at BRIEF, that by rewriting all the event handling functions it was very easy to convert it in an editor that no longer resembled the default BRIEF editor at all, but which behaved in whatever way you believed that an editor for program sources should behave.
So the text editor provided an extensive library of functions for handling text, text buffers, windows and files and a LISP-like language for gluing all these into a text editor. It also had a default implementation of a text editor, to be able to use it immediately, but that could be replaced partially or totally, when you had more experience and you knew what kind of behavior you want.
I assume that Emacs can be customized in a similar manner with BRIEF, but when I looked at Emacs it required a much greater initial effort than BRIEF, to reach a stage where you are able to modify it.
The default editing mode of BRIEF was very intuitive, at least for someone familiar with the IBM PC keyboard, so you could use it easily out of the box, before learning to customize it. Then, like most MS-DOS programs of that time, it came with excellent documentation, so it was very easy and fast to learn how to rewrite its event handling functions. With Emacs, both learning how to just edit texts with it, and then how to modify it according to what you prefer, is much steeper and more time consuming than it was with BRIEF.
It makes editing supported languages almost as pleasant as editing Lisp.
With how well the integration already work in existing editors, I'm curious how the UX can be improved further by an editor made specifically with AST editing in mind.
Comparison to Vim and Helix: https://ki-editor.org/docs/comparison#user-content-fn-1
> As you can see, there's no single logical categorization for these keymaps, they are either lowercase-uppercase, normal-alt, left-right bracket, or outright unexplainable.
Word, End, Back, Change Word and even Change Inner (, etc are very logical to me and I feel like I'm talking to the editor when editing. I get that it doesn't make sense when one has learned another way to do it, but it does make total sense you just have to make an effort to try and understand it.
It's like learning and always driving automatic then calling manual "outright unexplainable". You simply learned another way and are conditioned into believing that's the one true way. It shows the creator comes from VSCode (multi-cursor is a useless feature, just use s/search/replace and get used to macros and a whole new world will open).
Is there a good tutorial for some of these advanced text editing features?
In particular I’d like to get platform independent shortcuts / key bindings. I use both windows and MacOS daily, and it throws off my muscle memory for shortcuts like “go to beginning of line”
Just over a year ago I decided to switch to Neovim. The reason for switching was personal; I was struggling with what I'll call "clutter" in other tools and I wanted a tool that would reinforce, at least lightly, a mode of working that promoted focus on what I was working on, while making it easy to reference other files without loading up my editor with tabs and other visual clutter (buttons/menus) I don't care about most of the time.
I took the advice I seemed to bump into repeatedly: try out vim mode in my current editor before making the plunge.
I really struggled at first. It felt wildly foreign. All the shortcuts were nowhere near to the world I was familiar with.
As I was about to give up, I ran into some advice that was along the lines of "stop trying to memorize shortcuts and start thinking in terms of what you want to achieve" (words and motions in vim-speak).
Your example of [C]hange [I]nner is a great one; that one in particular was life changing. Sure there are some words and motions that do require memorization, but so many others just flow naturally. And once you start thinking in actions, it's easy to see how they can layer on top of each other in really elegant ways.
I'm not even here trying to tout vim-like editors, I'd wager there are many editors that have some semblance of this kind of interaction, but rather to reiterate there's a shift from a PoV of function vs. goal.
Again, I don't think this is "the right way" but rather one of many perspectives that works in context with the phenomenology of me.
Hey, one of the creators here, I actually daily drove Neovim for two years, before switching to Helix for a while, then finally Ki.
> multi-cursor is a useless feature
I was a Neovim macro user until I figured out how insane that was compared to multi-cursor after using Helix.
Multi-cursor was the first plugin I installed when I moved from VSCode to Vim because I was used to hitting Ctrl+d to select all words and then replacing. Does Helix do something different?
1) First I reach for <C-v> for visual block selection if everything is neatly aligned.
2) Next choice is %s/search/replace(/c if I need confirm).
3) Macros, and I love it everytime I get to use them. I just record the movements, copy what I need to copy, paste it where I need to paste it, and it's repeatable for every line or block where the *formatting* matches. And this is the important part, the words don't matter. I still feel like a wizard using them.
As far as I understand multi-cursor option 3 is a no-go without macros if the words don't match. But macros don't care as long as the movements translate to the same edits. How does Helix multi-cursor work that make macros insane?
It's hard to explain unless you actually try Ki, because it is a paradigm shift
I'll give you that using the AST to select the references is an interesting addition to multi-cursor, but I still don't see how they would be useful compared to my current workflow.
I also feel like macros are a more clunky and error prone way to do what substitutions can do. Almost never use them.
I think the same goes for multi-cursor, though.
Most of the times, shift means "bigger", and only in a few places it means "invert".
Examples of "bigger" are all the motions involving words vs. WORDS, where WORDS are a broader interpretation of words; "V" is like "v" but by lines, thus in larger chunks; "C", "D", "Y" do the same as their unshifted counterparts, but extended to the end of line; etc.
Examples of shift meaning "invert" are fewer, and all sound irrational to me:
With x/X and o/O, the shift inverts the direction left/right or above/below, but we don't have for example i/I to insert to the left/right of the cursor, and H is not the opposite of h, J is not the opposite of j, etc.
With n/N, shift inverts next and previous. Why not n/p? I know, I know: p was already taken for paste, but still...
Finally, t/T/f/F are completely mixed up. In my mental model, t/f sound like to/from and I'd rather use them to move forward/backward till (not including) a char, while T/F would be the "bigger" variants which include the destination char.
I've been dreaming of writing a plugin that surrounded the cursor in differently colored scopes. So instead of "next function" I'd be thinking "next blue" (blue being the color that functions are currently painted in).
This stuff reminds me of intentional programming (Charles Simonyi). That never really panned out but the premise with intentional programming was that programming is about AST transformations that are correct (sort of like refactoring) but all the way down to the machine code level. You get these increasingly more complex languages that are defined in terms of building blocks that are ASTs.
Refactoring of course is based on the same notion that manipulating code correctly at the AST level is a lot easier than messing around with text files. That came out of the smalltalk world originally which also had very fancy IDEs already in the nineties where everything (including the IDE) was something you could mess with. Refactorings were defined as syntactically correct transformation of one AST into another (rename, extract class, move function to parent class, etc.).
Charles Simony had a solution for what you describe as well. Just serialize stuff to text if somebody wants to "edit" and then parse it back when they are done. What happens to the text in between of course is the programmer's problem.
Refactorings preserved behavior.
Maybe that isn't what we wanted:
"A very large Smalltalk application was developed at Cargill to support the operation of grain elevators and the associated commodity trading activities. The Smalltalk client application has 385 windows and over 5,000 classes. About 2,000 classes in this application interacted with an early (circa 1993) data access framework. The framework dynamically performed a mapping of object attributes to data table columns.
Analysis showed that although dynamic look up consumed 40% of the client execution time, it was unnecessary.
A new data layer interface was developed that required the business class to provide the object attribute to column mapping in an explicitly coded method. Testing showed that this interface was orders of magnitude faster. The issue was how to change the 2,100 business class users of the data layer.
A large application under development cannot freeze code while a transformation of an interface is constructed and tested. We had to construct and test the transformations in a parallel branch of the code repository from the main development stream. When the transformation was fully tested, then it was applied to the main code stream in a single operation.
Less than 35 bugs were found in the 17,100 changes. All of the bugs were quickly resolved in a three-week period.
If the changes were done manually we estimate that it would have taken 8,500 hours, compared with 235 hours to develop the transformation rules.
The task was completed in 3% of the expected time by using Rewrite Rules. This is an improvement by a factor of 36."
from “Transformation of an application data layer” Will Loew-Blosser OOPSLA 2002
I think that's working against the way that a lot of people write code.
You need things like syntax highlighting and code completion even while typing something like "my_var = " on a line. A normal parser doesn't work here, and bailing out to make it the programmer's problem leaves the programmer with a lot of work
Ex: https://github.com/microsoft/tolerant-php-parser/blob/main/d...
If it’s all tree-sitter based though I think it ought to be possible to do this in EMacs too.
At this point you can just let your IDE's AI refactor any code by just telling it what you want to do.
Vim's j moves down.
Ki's j in line mode moves up...
Cannot figure how to create new file
I don't know what to feel
This, alongside the use of a variable means you can, in fact, build your own selection modes! Likewise, nvim does have a seperate select and visual mode.
This is primarily designed as an option to replicate the more traditional shift+movement selection and type to replace functionality of something like vscode or notepad, however it can be exploited as an additional layer for storing keybinds in a pinch
https://codeberg.org/alicealysia/ki-bindings.nvim
Unfortunately however, there's some shortcomings of neovim which make its control scheme impossible (at least not without some workarounds)
One of the biggest is ki's momentary layers feature.
Ki uses the kitty keyboard protocol to detect when a key is being held and when it is released in order to allow for unique actions when multiple keys are pressed simultaneously, and this functionality is a big part of what makes it so ergonomic.
For example, tapping c will copy the currently highlighted text, but pressing c and k simultaneously will duplicate the currently selected text to a new line below the current line.
While I'm currently chipping away at a pull request to introduce similar functionality to neovim itself, in the meantime, I'm needing to work around the issue by making a lot of concessions.