Over the past 21 months I’ve written a code editor from the ground up
edita.vercel.app
edita.vercel.app
I love these types of projects that happened because someone needed it and wanted to, and I loved that they came to us with the project in hand so they didnt have to talk down 100 but-why's like "dude why do you need yet another hobby editor, just use vim/emacs/whatever, i program in $LANG too and use these 37 plugins and 4 keymappings and it all Just Works".
I contribute to a popular IDE that I'm sure a lot of you use, and I'm still going to put down my setup and put a week all-in into this just to, if anything, spiritually put something out there, something positive, to validate this person's work. I see value in projects like these, and though it's hard to put into words I hope someone more eloquent echoes this sentiment in a more tangible way. This is great stuff, I wish more people broke the norm and just did-the-thing.
> Randomly deleting unit tests may seem like insanity to some of you, and in certain contexts it probably is — but writing your own editor is a particular endeavour, and if you embark on it I encourage you to do whatever feels right to you, refactor as you please, and embrace the chaos. Happy hacking!
I wish more people dared to buck the trends. Good on you, Gus.
I too, like to live dangerously.
It's so weird when you think about it. People don't delete unit tests because not having them is bad. So, they... don't write them? "No unit tests" is a state we can always get back to in a seconds. Don't be afraid to leave it.
I've been wondering if there is some analgory between programming editors and armoured military vehicles.
The backend guys might need a full-on tank (Abrams/T90) - an IDE/hyper-custom vim. The front-end infantry might need a lighter weight but highly mobile IFV (Bradley/BMP) like VSCode/whatever.
It's possible the IFV/artillery SPG/etc vehicles can also be built on the body of a tank to "save money" and time. Modularity is important right? But it also creates frankensteins like the BMPT [1] which is an IFV built on top a widely available T-72 tank platform but costs 2-4x what a normal IFV/BMP costs [2] and lacks the benefits of either an IFV or main battle tank.
Ultimately it has little to do with the inherent generic capabilities of the base editor/platform. It's about designing the platform around a certain audience and skillset, then building a community around it's mission. Then iterating on that particular usecase.
I was a hardcore fan of Vim for the longest time but I've come around on VSCode as the essential default for frontend/Typescript development which I do. Sometimes extensions/plugins aren't enough. You need an editor built around an idea/community that fully embraces it. Much like programming languages.
[1] https://en.wikipedia.org/wiki/BMPT_Terminator
[2] "Price of the BMPT is equivalent to two modern Russian main battle tanks." https://www.military-today.com/tanks/bmpt.htm
This aside, I agree with your point - the best articulation I can think of right now is the human equivalent of "do what I mean not what I say". Where sometimes the collective intent gets stuck in an ideological rut and can't become what everyone wants it to be for lack of expression.
And to your point, I couldn't imagine developing Java without Intellij. It's like they are made to work together.
However, I love the vim community and how they port VIM bindings to almost any IDE(or most software that works with text and allows for some extensions). Same way to deal with text across OS/Machines and software feels very liberating.
Solid advice in the early-ish days of any project.
It's not like your code was perfect before the refactor - you just hadn't found some of the bugs yet. You will have bugs after too, but now you have more experience and will probably find most of them quickly. Don't get too attached to the maybe-bugs you currently have.
Also:
>When you’ve got a while (true) loop that searches a tree, it’s gonna throw you an infinite loop every now and then.
A surprisingly effective strategy I've found for development purposes: never have infinite loops.
Nothing is actually infinite. You think X might loop 100x? Put a limit of 10,000 ("absurd and impossible"), and crash your debug builds when it happens. With enough of this, you'll get hits, and they're extremely effective at catching pathological cases that you didn't realize could happen. "It's a bit slow" is often too hard to notice.
This feels like a lot of line noise for those rare-until-you-hit-them bugs. Just capping a loop with a fixed iteration size deserves a comment as to the rationale. Do you do it everywhere? Just IO?
The idea has merit, but in practice, I am not sure where I would implement it. Plus, there would always be the temptation to remove that do-nothing code.
Yeah, but if doing so can eliminate an entire class of bugs...
...you know something? I think I'm starting to see where those Rust guys are coming from.
It's not 'do-nothing' code, though. Isn't the temptation just the same as the temptation to remove 'do-nothing' exception handling or 'do-nothing' unit tests?
def at_most(num)
num.times { yield }
raise "Should never get here" # or "binding.irb"/"binding.pry" to drop you into a repl, perhaps contingent on a command line flag.
end
and do:
at_most(5) do
.. logic
end
If the number isn't obviously absurdly high, I agree it'd be worth documenting why the precise number was chosen, but then it might be worth documenting why as an indication of why you thought an infinite loop was appropriate anyway.It's really not much noise, and it serves as (mechanically checked!) documentation of the expected size of a loop. In some ways that's an improvement, since it's just a loopy assert.
And no, it's a tool. Use it where it makes sense.
Also, "personal projects don't fail until you decide they do" is a great insight :)
I really like to keep my code "elegant" and so I tend to constantly refactor it (which also makes a typical refactoring step small, and thus not a chore).
At work it's sometimes a hindrance: I tend to overdo the clean-up stuff; for me it's hard to resist a temptation to have even simple throw-away code clean and maintainable, and it is difficult for me to cut corners when working on something urgent. Of course sometimes throw-away code is not thrown away and is maintained for years and years, and cutting corners accumulates code debt (which you later on pay with interest), but overall I tend to be slow with small projects, although reasonably fast (if you include debugging time) with large ones.
But when working on a personal project, and can tidy the code up to my heart content! It actually helps to keep up my enthusiasm, not the other way around.
I guess different people have different preferences and predilections :-)
I guess you can have a limit on the loop, have it break out to make sure you're still listening to interrupts do some logging and go back to looping, but in an environment where interrupt signals are native, an infinite loop can and is fine.
Glad I got my bikeshedding allowance out for the day.
I agree it's usually fine to use infinite loops for high level event loops, as it's usually easy to test that you can break out of them by triggering the right event (though sometimes it might be worth thinking about pathological cases if you can do something about them, such as whether something is wrong if there's no interrupt for a long enough period; and you can also prevent writing an infinite loop even for event loops by making the break condition whether a suitable "exit"/"restart" event has been received - very few things should be without one or the other of those two even if intended to run forever in normal circumstances).
The advice, I think, is more important for algorithms working on input data that may well lock up the application if the logic is wrong. E.g. a sort should eventually complete; if it takes more passes through an outer loop than the theoretical worst case for the algorithm, something is wrong (be it your implementation or your understanding of the worst case...). A habit of estimating worst cases and treating them as invariants to check and make assertions about can be useful.
For example, parsing of an XML document naturally resembles an event loop, with each token being an equivalent of an event.
Although usually there is a natural end to this loop, so you have not "while(1)" but "while (not EOF)" or something, which helps avoid some bugs, but I believe a compiler turns it into an infinite loop anyway, and breaks it on these conditions.
This also made automatic minimization of bugs much easier and faster.
One thing that other (applicable). projects dont seem to do that llvm and gcc were quite good at was using automatic tooling to minimize test cases for bugs. Find a large test case causing an issue. Run automatic minimizer and leave it alone for a bit, come back to minimal test case that reproduces your issue
Wait, what?! To each their own, but you'll pry my well-maintained collection of unit tests--which describe all _defined behavior_ in my application--from my cold dead fingers, thank you very much.
- The test suite runs faster
- Less time is spent maintaining the test suite
The argument wasn't whether tests should ever be deleted, but rather an encouragement to delete them as soon as you feel confident in your code. Personally, if a test is still relevant then I will maintain it until the behavior it's testing is no longer part of the application. And often that's a zero sum situation, because now you need to test the some new behavior.
A failed test is literally your test suite saying, "Remember this part of your application that you promised would behave a certain way? Well, surprise! It doesn't behave that way anymore." There is value, if nothing else, in knowing what the implications of a change are, especially as the system grows. So even if you're okay with a change in your application's behavior, I'd argue that tests are the best way to be aware of what those changes are.
Tests certainly can be valuable for verifying the scope of changes, but they're not the only way to do this. I could imagine supporting the process of delete-as-you-go testing if you could convince me that whatever you are doing to identify regressions works just as well as the test suite would.
Of all the languages I would choose for a desktop app, it ranks pretty far down, yet I am clearly in the minority.
But I’m not here to show my age or biases - I am genuinely curious. What am I missing that makes JS/node (or is that a redundancy? Is there any other choice than node for JS local development?) such an attractive choice for local development.
Over say java / swift / rust for static languages or python / closure / julia for dynamic?
I realize though there are not many frameworks for cross platform development. If you do want a native language you'll probably have to use QT. There are more out there, but most are based on native widgets which have their own unique behavior which makes them difficult to abstract.
It seems to me that these electron web apps provide this 'leak free' abstraction.
There's essentially no technical debt in choosing these technologies because if you start with a web app and then want to make a "desktop app", there's Electron. Want to make a mobile app? React Native. There's even React Native for Windows and macOS for a more true desktop app with a native UI. You can also run your JS on the backend with Node, and even inside the database (PL-V8 runs V8 for stored procedures inside PostgreSQL, aside from obviously CouchDB or MongoDB and others). Heck, you can even write an app for the Xbox or build out on several IOT platforms using JavaScript!
Is there any reason to create the same thing in 4+ different languages? Are you going to rewrite your frontend in C#, Kotlin, and Swift? Are you going to deal with different language models and incompatibilities? Rewriting the same exact functions over and over, and unit tests for every single one, keeping in mind the subtle semantic/scope differences between each language--and then realize you still have to serialize/deserialize to JSON to talk to your backend and other services and just juggle JS data models between all these languages and ecosystems.
Choosing JavaScript and React today are the sensible default where you won't code yourself into a corner, is what I'm getting at. And you can also now choose TypeScript or ClojureScript in all such cases as well.
Electron leads to high latency and slowdown once your editor grows big enough- even on 8+ core machines. We have so many electron based editors at this point, including incumbent VSCode....
Any large application who extends the number of features forever leads to high latency and slowdowns, unless you structure things as extensions/plugins and load them ondemand/allow users to disable/enable them.
It has nothing to do with Electron, same thing would happen with a application written in C/Rust. Might happen faster with JavaScript, as JS developers tend to have less focus on performance, but it is less about Electron vs the World and more about who the developer(s) is.
That's gonna hurt
>When you’ve got a while (true)* loop that searches a tree, it’s gonna throw you an infinite loop every now and then. Fortunately, the more times this happens the better you’ll get at lightweight debugging strategies, and the more familiar you’ll become with the algorithm and the data structures
I like your style, kid.
"Just do it, hack! I approach code like games, rush deep into room, trigger all NPCs, die, after respawn I know where NPCs are." --@bkaradzic
> Use it early
This was key to me. If you're going to wait until you have a perfect editor to use it, you're better off spending your time on something else, because it'll take a really long time to get there if you ever do.
I started working on my editor because I was endlessly tinkering with my emacs config and realised I could write an editor in fewer lines than my config...
You need to have a reasonable clear picture of what an MVP looks like to you. But having as a goal to use it early can also drive design.
E.g. in my case the first thing I did to enable this was to have the editor serialise all open buffers to JSON and write a dump regularly. If I didn't expect to use the editor until it was stable, I might not have. But I came to like being able to start the editor after a reboot and have all of my buffers as they were.
I later also moved the buffers into a separate process using DrB (the editor is in Ruby). Combined that made it near impossible to lose changes as DrB makes the backend near crash proof (exceptions are forwarded to the client), and the checkpoints/dumps deals with the last little piece of risk.
The day that was finished, I switched 90%+ of my editing to my own editor even though it was still buggy and lacking in functionality.
As a result I as of writing have 2199 open buffers, some dating back a couple of years. I kill some now and again - e.g. if I open a particularly large file, or something sensitive - but by default I just stopped caring as the dump is only 66MB.
> If you’re using the code editor to write itself, the most pressing bugs and missing features will make themselves apparent to you for free.
This is a big deal with using my own editor. I have a "roadmap" of things I like the idea of, but "what is painful right now" trumps all of it, because I don't have any users (nor do I want any - I'm slowly splitting functionality into gems that I'm happy to deal with issues with, but the github copy of the repo of the editor itself is wildly out of date and heavily dependent on my personal setup, so likely won't even run for anyone else)
It also means I can follow the "refactor when it seems like a good idea" advice without caring if the editor is half broken for a while. I have bugs in there that are clear regressions that I'm ok with working around but that I'd never allow myself to commit code for if I was writing this with the expectation of other people using it.
I am also curious what advantage you found in putting the buffers in a separate process. Is it just that a crash in the front end doesn’t cause you to loose changes? Why not just save at regular intervals instead? Is the idea to persistent undo or file revisions?
I am also curious what you thought was a limiting factor of Emacs so that you found it easier to write an editor from scratch.
I started out with Femto [2] and ended up quickly gutting it (there's next to nothing left now), and as an example of a minimalist Ruby editor, Femto is perhaps a better starting point - it's certainly both cleaner and smaller.
I've started breaking a bunch of functionality into gems (termcontroller, file_writer, ansiterm, editor_core, keyboard_map) of various stages of usability for others, and my goal is to over time eventually be left with the editor as a tiny shell around reusable components + my personal config, but as per this article, I plod along and fix the things most pressing to me personally and nobody else so who knows when that will happen.
As long as it keeps being useful to me, that's what matters. In that respect it's also a meditative experience of sort that takes my mind off work and having to work in a more professional manner.
> I am also curious what advantage you found in putting the buffers in a separate process. Is it just that a crash in the front end doesn’t cause you to loose changes? Why not just save at regular intervals instead? Is the idea to persistent undo or file revisions?
I also save at regular intervals, so you're right that if that was the only consideration I probably wouldn't have bothered.
But the editor is a terminal application, and I use a tiling window manager (bspwm), and so I figured that if I could spawn another editor instance and have my wm open it in the right space and with access to the same buffer, I didn't need the editor itself to have any understanding of windows itself. Here's "split_vertical" that opens a second view of the same buffer:
def split_vertical(buffer_id)
system("split-vertical 2>/dev/null term -e e --buffer #{buffer_id}")
end
[EDIT: Should also clarify here that "term" is a wrapper for whichever terminal I currently favour, at the moment mlterm, and "e" is an alias for my editor; if I ever were to try to make this usable for anyone else, there'd be a whole lot of cleanups to make - the biggest concession towards this I've made has been to recently move most dependencies on my environment into a separate file]split-vertical is a helper (I split out a handful of things that depends on external tools) that just runs "bspc node -p south" to get bspwm to halve the height of the current window and "swallow" the next window being opened into the space freed up below the active window, and then exec's its argument. So meta-2/meta-3 works roughly like ctrl-x+2/ctrl-x+3 in Emacs (EDIT: fixed; already forgetting the Emacs keybindings...), except the frames end up in their own windows, running their own client.
Had remoting the buffers been a lot of work, I wouldn't have, but the the Drb specific part of the code is <100 lines and the combined effect of the resilience (if an exception is thrown in the server code, I get thrown into a Pry repl in the client) and the extra abilities it enables makes it worthwhile.
Similarly, I've avoided implementing my own directory handling, and just farmed that out to a wrapper that currently uses rofi, same for theme selection (for syntax highlighting), buffer selection, etc.
The minimalism of delegating all of these things elsewhere appealed to me.
> I am also curious what you thought was a limiting factor of Emacs so that you found it easier to write an editor from scratch.
Strongly disliking Lisp was high on the list (I like the concepts; hate the syntax intensely). Also, it wasn't so much a limiting factor, as realising that my Emacs config file was larger than toy editors I'd played with in the past, and a minimalist bent making it appeal to me to go for something smaller. Also seemed fun (and has been)
https://codepatterns.vercel.app/
There’s one column missing: whether the query is purely syntactic or has semantics as well.
In particular, JetBrains allows not only searching for all _expressions_, but also, eg, constrain expressions to be of a specific type, something which isn’t possible with just a parser:
https://www.jetbrains.com/help/idea/search-templates.html#ty...
vscode and modern IDE now take 1-2 gb of ram for medium projects. What has caused so much resource usage?
Might actually work with Java-like language (JVM bytecode is very close to Java AST).
(Scroll up in that thread for AST inspiration)
Some of these ways of working will be brutally beaten out of you in most dev teams unfortunately, whether they're effective or not.
Yay cargo culting!
I love vanilla JavaScript, but I always try to switch to TypeScript if the codebase becomes larger than 100-200 lines of code. Or otherwise I may regret, and I know it too well from previous experiences.
When I hire devs, I don't care much about their previous stack experience, but if they've actively chosen JS over TS, it tells me their philosophy/priorities are completely backward.
This is a superficial take. TypeScript is not free. You are adding a compilation step that takes time, slowing down feedback in certain workflows, as well as a rather large dependency. It certainly can be worth asking why they chose JS given some of your perceived benefits of TS but not using it doesn't mean that their philosophy is backwards.
A one sentence comment about how the author is bad at programming says more about you than the author. People have been writing text editors like that for decades. They aren't bad programmers.
Also, using esbuild to transpile your TS will not actually check the types. It just transforms it back to JS. For typescript, you really need the official library which is slow and bloated.
It is free if your IDE uses the TypeScript language server and gives you errors integrated with your code editor. You don't have to transpile if you don't want to. You can, as you said, just use esbuild to strip away the type information.
> For typescript, you really need the official library which is slow and bloated.
See above. You absolutely don't. And it's not slow or bloated at all. It has incremental builds, which are essentially instantaneous after the initial build.
Indeed it will not check your types, but as stated elsewhere your IDE can do this for you. This is actually beautiful because it allows you to get correctness feedback on your code but your code will never fail to run due to arbitrary type complaints. If you later feel like you want the full correctness of TS, maybe during CI/CD, you can run the slower compiler and get a gatekeeper telling you if you are doing something fishy.
IMO every JS engine should just consume typescript (without type checking). AFAIK TypeScript is designed to be easily consumed by JS engines. It's a pity that V8 ignores that property.
I don't think it does, it comes with a TS compiler built-in, and "can run TS" by compiling said TS to JS and evaluating that. Deno is still using V8, so unless Deno has changed recently, it does not "consume TS" and no mainstream JS engine does.
There likely will never be an actual TypeScript runtime because that doesn't make sense as a concept, and it's against the philosophy of the project. TypeScript is metadata for devs.
That's fine, doesn't make the truth any different that it doesn't actually read TS without compiling it, as parent said.
> There likely will never be an actual TypeScript runtime because that doesn't make sense as a concept [...] TypeScript is metadata for devs
It does make sense to have a runtime for TypeScript directly. There is bunch of optimizations that can only happen after the runtime understands the type of the values, and if you keep switching the type of the variable value it'll be able to do less of those optimizations.
So you could probably end up skipping a lot of inference if the types gets shipped to the runtime as well as the rest of the source, vs just the source.
I've been using TypeScript in server-side, client-side, and hybrid projects for 8 years. Since then, I've inherited JS projects and felt like I was being forced to use a rock when I was used to having hammers. My take isn't superficial.
> You are adding a compilation step that takes time
The entire project is compiled only when testing or deploying. As far as testing goes, having static types dramatically reduces the number of times you have to run a program before shipping it. The net is that there's less time waiting for builds, not more.
And anyway, most projects compile in 2 seconds or less. There is a huge net time saving no matter how you look at it.
> slowing down feedback in certain workflows
This is incredibly vague. Which workflows? And why does it matter in those workflows?
The majority of feedback given by TypeScript is during the editing process, and that feedback is instantaneous because it comes from the language server and doesn't require recompiling.
> a rather large dependency
No. It's a dev dependency and doesn't ship with your code. You have to install it locally at some point, so I guess if a few MB of space is important to you, it's a problem. Otherwise it isn't.
O(1) setup cost for O(n2) productivity gains.
I would take Emacs Lisp over Typescript any day, but that's just me.
I think starting in JS and then migrating to TS is a very good strategy that lets you develop fast but once you figure what you needed to wrtie let's you get additional assurances and conveniences about what you wrote.
No sand or gallium containing stone was transmogrified in this story.
"I wrote a code editor from the ground up. I started with notepad and stopped there."
> App threw an error during load
> Error: --start-args argument required to separate command-line args from node/electron paths
> at filters (~/progs/edita/build/electron/mainProcess/utils/getArgs.js:13:10)
after `npm start`.
>>
- ...
- Perfecting smooth and ergonomic interactions over shipping quickly.
- Providing rich and configurable features over simplicity or minimalism.
- A visually pleasing and quiet interface (over keeping up with design trends).
The world would be a much better place if that was the case.
What is about coffee in your actual nostrils that wakes you up more than the full cup itself?
Not the person you're replying to, but as a food and beverage expert, I'm confident most beverages will yield more powerful emotional and physical responses and quicker absorbtion of many chemical components when taken nasally.
But if you are looking to give this to the world, the lack of planning and emphasis on testing is less than ideal.
It wouldn’t hurt to play with some editors and write a spec so data structures and algorithms can be planned appropriately. After all, there are tons of great examples to get inspiration from!
I like to take a mix of slow and fast approach. While some cases demand test-driven development, in other scenarios the test cases can follow the user demand. I like to build test coverage slowly depending on most used parts of the code. So they coverage catches up slowly but at the same time i am not spending time on test cases for things that don't get used at all.
This means, I would prefer releasing features in small batches and as the features start being used, I start improving the coverage. While this may not work for all the teams or environments, it is one approach to build early stage products. which I mostly do.
What historically doesn't scale is trying to get everything "scalable" and "extensible" from the get go - most ambitious projects like that get abandoned because they get the initial designs wrong and its too hard to change, or because there's just too much effort to get to usable state...
The difference is stark.
I've worked in multiple 20k+ LOC projects with other people and everything (teams of 10 or more people are good enough to you?). And 20k+ is not even that big, it's not like it offers any great insight regarding software development.
You can write simple and maintainable code for a simple product which isn't scalable or extensible.
I rather think, it has to do a lot with momentum. If you think hard about every step and every change, you will be so slow, that a competing project that just focus on shipping useful features, will get so much traction, that they overcome their wrong design decisions with more man power and after a while, so many people use it, that they continue to use it, despite its flaws.
Otherwise we wouldn't have javascript as the most widespread language for example, with monumental efforts invested in fixing the initial flaws and making it do things, it was never designed for.
When I was starting out years ago, my passion project ended up on HN with a very dismissive comment similar to yours - and it almost turned me off of my passion project of 4 years.
I looked at their package.json, they pretty much pulled in the standard library (if there ever was one) of JS. Like lodash and that's about it. Not once did they claim "zero dependencies" or "running on my own OS" so your comment is not only dismissive, but wrong.
I for one would encourage people to experiment with building new editors, not dismiss them. Otherwise we end up in a world where editors are built only by large corporations.
The title literally says "Over the past 21 months I've written a code editor from the ground up". The linked blog post is titled "How to Write a Code Editor from Scratch in 4 Months".
I don't know what you would consider "from the ground up" or "from scratch", but in my view that means no external libraries, if something isn't in the standard library I code it myself. That means that someone only needs a compiler and the standard library to build the application, rather than having to either use a dependency manager or (worse) track down and build the dependencies themselves.
If you wish to make a text editor from scratch you must first invent the universe.
Seems like you arbitrarily draw the line. rustc but not cargo?
Even if you don't use any libraries, cargo is a useful tool. And if you reject tools, what editor/IDE would you code in?
I'm not in the construction business, but I guess if you build a house from the ground up, that means you're not building on top of some existing structure, but you might still be using prefabricated elements.
I'd put the line here: whether ground work is used that was laid for editors, as opposed to general programming. If the thing that they are making (an editor) does not contain editor-specific prefabricated parts, then all the editor-specific work was still made from the ground up.
That's just silly.
apt install emacs
Or should we, perhaps, consider “from the ground up” and “from scratch” to mean something?For example, for me someone writing a new editor not using advanced components such as existing programming editor components counts as "from scratch".
They don't need to use magnetic needles to shift individual bits on HDDs.