Don't Write a Programming Language

I'll take the bait and provide evidence on why you should write a programming language.

First, it's technically very difficult, but you will gain deeper insight into the art of the craft. So, if you are a TC chaser or career minded person, then spending half a year writing a language will help you master the coding aspect of the game. I've started many languages since I started college, and each one was instructional. (I'm now 40 and an early retiree)

Second, it's fun.

Third, it may turn into something new. If some people don't write a new programming language, then we are stuck with what we have. This advice basically admits that the status quo is good enough.

The authors saying that the language, as a project, is a lifetime appointment? Well, this reveals everything. I believe if you want to do a programming language, then you must be willing to invest at least a decade or two.

So, here, I am at forty preparing to launch a SaaS around a language that I designed ( http://www.adama-lang.org/ ). The kicker, I believe, is that a project like this requires wandering the desert alone for quite a while.

I'm preparing to launch, and I just started to load test my shiny new production cluster. Low and behold, it sucks. Fortunately, I have a tremendous number of dashboards and isolated it to how I'm interacting with RDS. I've got my work cut out for me which I'll write about.

However, I have a potentially interesting business precisely because I evolved a language which solved a niche use-case. The number of problems that I have had to solve up to this point is not for the faint of heart. Life and reality are harsh mistresses.

So, maybe, yes, you can save yourself some heartache by not writing a language. Perhaps, a better way to think or phase this is "Writing a programming language is a lonely affair that will most likely end in tragedy after a long death march".

The problem with writing a _general-purpose_ programming language (as a solo-developer), is the fact that writing the language (i.e defining the syntax+semantics, writing the parser/compiler) is only half the battle - the rest of your time is spent on less technically interesting QoL features, like an IDE, standard library, debugger, syntax highlighting etc. which are nonetheless vital for making the programming language actually usable. (I still think writing your own PL is a useful exercise just to see how the sausage is made, but it should be made clear that there is a world of difference between a toy language and one that people would actually want to use)

It's for this reason that nowadays I rarely write my own "new" programming languages from scratch, but rather embed them as DSLs inside existing programming languages using macros. This way you get the full ecosystem of the host language for free, leaving you more time to work on designing the language to suit your needs. If you combine macros with clever uses of GADTs you can even get the host language to "type-check" programs written in your DSL. OCaml and Lisp/Racket are my favourite host languages to use for this approach.

This is so strangely sad. Go ahead and write a programming language. Have a weird hobby you clank away on that might not see the light of day. If you burn out on it, that's one thing and that's okay; you've exhausted the medium. Move on to something else you enjoy. Grow for the sake of growing.

Just because something hasn't become big doesn't mean it's a failure. I've labored in dark early hours over paintings and music that will never see the light of day, but it's a vital part of how I express and cultivate myself. My journal.

99% of art is unknown personal projects. Embrace yours. Why be bitter?

Perhaps this is good advice, but perhaps even better advice is: Don't write a Turing-complete DSL to solve FizzBuzz at a job interview.

(Warning - comic gold, get a tissue) https://aphyr.com/posts/353-rewriting-the-technical-intervie...

I have also been working on a programming language, but longer than the author of this piece (since 2011), and the author is correct.

Designing a language consumes you; you never see programming the same way again. Whenever I write C (I'm implementing the interpreter in C), I get annoyed because I have to work around C's warts to do what I want, and that is only happening because I think in my language.

And the design never stops. As mentioned, I've been designing mine since 2011, and I'm only now writing an interpreter after two failed attempts to bootstrap a compiler.

So the author is correct: don't write a programming language unless you are prepared to be marked and scarred for life.

A programming language is a tool. And like all tools, the situation is very different depending on if you are writing a personal tool that solves your problems or if you are trying to solve other's problems.

I used to lurk in a programming language design group. I joined it because I wanted to "solve everyone's problem" with a new language because I had some success with solving my own problems with a previous language (which is not for everyone; it's a Forth dialect).

For few year I saw some people theorizing and analyzing with little practical results. It's fascinating how people can know about "analysis paralysis" and still fall for it. Meanwhile, I lost interest in my own attempt and returned to my Forth dialect.

Today it powers a few programs that run 24/7. And it is also my goto scripting language. It's not available on Github/Gitlab/whateve because I'm too lazy to do something that will certainly have zero benefits for me ( however curiously, it is half-documented because writing docs helps with spotting silly decisions sometimes).

It is true that you are never done with that sort of program, there's always something to improve or fine tune. Also, since you are the sole user you cannot rely on a ecosystem and just "import their.stuff".

But I think the great mistake is really to start making a tool, a programming language, with no plans on what to use it for; or worse, expect others to use it. The Internet is indeed littered with born-dead or short-lived languages. They are solutions without a problem to solve.

It is true: it's a deep pit you'll enter. But the conclusion is wrong. If you feel it's fun -- then do write your own programming language. Even if you won't finish (which is likely). But you'll think about things that are a lot of fun, and you will have more clarity about what you do not like about existing languages, and why. And you'll learn that there are things you need to solve that you did not think about before. That some things cannot be combined, and that you will have to choose.

It is a very good experience, there is a lot to learn. As long as your goal is not to get it done quickly...

Who cares? I'm not writing my programming language for you: I'm writing it for myself. It's about the journey, not about the destination. (Btw, the author is right)

I started creating a language about fifteen years ago.

It started as a project to learn C: implement an interpreter in C that lets you interactively call C library functions and system calls.

But, little by little it become my own implementation of my own dialect of lisp and the language I use for almost everything.

My only claim for the language is that I like it, I like it a lot.

I guess I will release it at some point, to the indifference, no doubt, of the world, but that's not the point. If you design and build your ideal house, it doesn't matter if no one else builds the same design, or even appreciates it, as long as you love living there.

I disagree with the author's view that any project that isn't used by other people is a waste of time. Does that mean that all homework projects in college are a waste of time? Or that beginner programmers should skip writing simple programs and jump straight into writing a sellable product? Of course not. The point of those projects is to learn, not to make anything that will change the world. If a project is enjoyable to you and you learn something useful from it, then it is not a waste of time.

The author mentions that writing a programming language entails writing your own GUI (which language doesn't need an IDE). Surely writing a plugin to an existing IDE is what most people would do in that situation, and I can't see many arguments to the contrary. This alone made me question the author's advice somewhat. If they are such a perfectionist as to care to have a custom IDE for their language perhaps a more reasonable person wouldn't get so deep down the rabbit hole

I did design a language and it is still being used. I did it as part of my job at that time. It started as a simple query language inside the application we were developing. One by one features were added. Some colleague also decided to used it program certain functionality of the application, which caused more and more people to use it.

I implemented it with a parser that I developed myself. So, actually, I fell in the trap of developing a language myself, not a programming language, but a developing my own language developing tools. And I have doing it for more than 20 years now, on and off. I do recognize the self promoting bit. (I am kind of doing it right now.)

These all sound like pros... A project that makes you intimately familiar with a huge breadth of deep, foundational domains and that forces you to make opinions about esoteric concepts, all while pushing you to go deeper.

I find writing a library is usually enough for the types of problems I encounter. You can even design it to be language-like using fluent interfaces; take FluentAssertions for example: `actual.Should().StartWith(expected);`.

Beyond that, I tend to reach for:

- A config file. Can the problem be solved declaratively through a config file that a program reads and bases its behavior on?

- Templates, generators, and macros. Along the lines of syntax, would templates or generators solve problems with an existing GPPL?

- A domain-specific language. Does the problem lend itself to composable patterns or statements? Could a DSL help organize and simplify them?

- A transpiler. Is the syntax of an existing GPPL the problem? Would syntactic sugar be enough?

Of course, there are plenty of other reasons to write a new GPPL, like for your own learning or just as a hobby. Just know what you're getting yourself into.

Finally a bit of sanity!

The same goes for an OS or a game engine or any other large scope (Or collection of small scope) tool.

The other people who kinda care are those similarly engrossed in their own project, meaning... they do not actually have time to care.

My first priority with computing is to get rid of all the custom stuff. I won't even think about refactoring or any other improvement till I've weeded out all the handwritten reimplementations of library functions, made sure that there's no ad hoc string.split() based data formats that should be JSON, or UDP packets that should be MQTT.

Honestly, I don't even like that macros exist. I understand that in low level languages sometimes one needs metaprogramming, but it's more of an unfortunate reality than a cool feature.

I’ve been spending a lot of time myself designing and prototyping my “ideal” language (mainly to address gripes with other languages) and I totally agree with the OP about it being a near total waste of time. But at the same time, albeit being a gargantuan amount of work it helps you learn alot about programming up and down the stack. Good language design requires lots of thought, devotion, trial and error and self moderation to come down to something consistent and coherent that’s not a mess of all the cool little features you thought it would be a good idea to put in.

> Programming language design is at the same time the most interesting technical challenge you can take on, while at the same time being for the most part an utter waste of time.

Yeah that's a hard disagree for me dawg.

But when you write a programming language understand that you're writing it for yourself, so that you can look at other programming languages and understand the tradeoffs.

Just the process of going through "holy shit, this thing I'd like to build is going to be so much harder than I thought" is useful. Particularly if you come to understand that some things you want are at odds with each other and you can't build it the way you thought you could. Then you can look at other languages and understand that to have feature A they had to give up feature B.

Give up pretty quickly on things once you start to get the feel that they're pretty intractable and you start to hate thinking about them. If you enjoy something in particular then jump down the rabbit hole. Understand that you're going to wind up with something only 10% done that nobody will use.

Then maybe when you learn a new programming language go write your language in the new language to learn it. Wash, rinse, repeat to learn new languages. Get good at rewriting your toy language.

Writing a programming language is easy, when compared to making an actually useful programming language, which again is easy, when compared to actually maintaining and developing a programming language.

My heuristic is that the difference in work required is around three orders of magnitude between creation, usefulness and maintenance.

That being said, I think every programmer would benefit greatly, just by attempting to write a language.

> Don't Write a Programming Language

Is why we've arrived at using JSON for declarations instead of a nice domain-specific language. So while I would agree that you almost certainly don't need a "programming language", you should definitely consider a DSL over just using JSON.

I created a language without lots of trouble. The trick was low expectations. I named it “lila” (limited language), so they were right there in the name. It did very little, it had no compilation phase, but served its didactic purpose. It showed how to build a language using ANTLR.

I would rephrase this. First do you want to design a language or implement one? Usually you will want to do both, but where are your motivations and interests? The post starts with a desire to design one but then seems to have spent the most time on implementing it rather than iterating its design. Perhaps it became a sort of vanity project, idk.

Secondly, don't write more than you need to. Look for existing 'narrow waists' that can save having to build much of it. E.g. target a VM ecosystem, use language server tools for editor integration, etc.

It's good to have an idea what you want to do and why, but above all as others say do what you enjoy and learn along the way.

The same arguments could be applied to almost any field of research. If you get to spend your time doing something that gives you pleasure and challenges you, then it's worth that time. If it results in something that is also of use to others, all the better. There are countless examples in history of how people have worked on something, often despite lack of understanding or even rejection by others, and that very work later became very relevant, often many decades later. You could have spent the same amount of time watching TV instead of writing a programming language. Wouldn't you have found that an "utter waste of time" as well?

I never really got into it that deep: I thought i should enjoy it, but after writing a small toy language I realised I don’t. What I have realised is I get much more joy from building things than building the tools but that is just me. Im having much more fun knocking up stuff in nextjs! I am glad there are people who love building the tools and frameworks though such that I am spoilt for choice.

Most people will not need or want to design or implement their own language.

However, it is not as hard as the author suggests. First, there are excellent tutorials, textbooks (Wirth, Dragon book) and tools available (lex, yacc, LLVM, CUP, Jflex, Lemon, ...).

You should give it a try just for the fun of learning.

You will not often _need_ to do it, as most capabilities can perhaps be realized as libraries (including network and mobile functionality mentioned by the author).

Once you design something useful, other people will likely chime in and advertise for your, write documentation etc.

Most complex software system incorporate compiler techniques or Domain Specific Languages (DSLs), for parsing configuration files or as a basis for building higher-level functionality (AutoLISP -> AutoCAD, ELISP -> Emacs).

I can feel the author pain, even though I didn't invent a programming language, I just wrote the interpreter for one (specifically a Lua interpreter written in C# - moonsharp).

Why I did it: in 2013-2014 I was SUPER bored and burned-out, I needed a challenge and I picked that one. Oh, I was wrong.

The GOOD: the challenge has been fun, I learned A LOT. I learned how to use ANTLR, and then I learned that ANTLR was getting in the way and now I'm wary of ever using a parser generator at all, for sure not that one. I learned how to write a recursive descent parser, even if I did it by reversing engineering (i.e. mostly copying) the original rd-parser of puc Lua. And I learned that no matter what I do, what I find the hardest is respecting the correct operator priority in math expressions. I know that shouldn't be the hardest thing, but brains are fun. It also ended up getting used. It was used in the OnePassword client for windows, and it's for sure used in the Shenzen IO and TIS-100 videogames. I'm sure there are many more users using that. It's stuck somewhat in between of "nobody uses it" and "it has enough traction that the community will take it forward and you can forget it even existed", and that, turns out, is not a great place to be.

The BAD: I picked C#, as that's the language I was using at the time and I had an interest in seeing it used in Unity. Big mistake. Don't get me wrong: C# is a great language and if you're planning to base your company on it, I don't see big reasons not to (I wouldn't, probably, but every language has its pros and cons). But C#, and in 2014 to boot, was a mistake for such a project. First, however multiplatform C# became, a lot of development is still Windows only (at least, if you want to target the main frameworks). And second, 2014 is precisely the year when C# started changing radically with the core / standard frameworks. The situation now is probably more stable, but in 2014/2015 it was a mess, and half of the reflection frameworks were broken to say the least.

The UGLY: I just didn't consider how much punches life can hit your face with. I changed job, twice. Changed platform from Windows to MacOS, to Linux, to MacOS again, now I program for multiple platforms in Rust most of the time, and my interest in C# is close to non-existent. Family members passed away, new kids were born, situations changed, and life hit me hard enough. Energy got focused on daily jobs, or on other side projects. Every other side project, by the way, has been met with humongous amounts of guilt, because I'm/was leaving moonsharp users (however few they might be) behind.

So where I'm now? I tried to delegate the project to other people, but I'm not effective at it (maybe they are, but in reality, I'm just absent). I tried to get back active into the project but failed a couple of times and now I fear doing it again and creating expectations. Sometimes I feel like contributing something, but I can't even get a proper dev setup running. I don't remember details well enough to not break something when fixing major bugs, nothing major but usually it goes over the amount of energy that I would like to spend. Sometimes I read some question on MoonSharp, and I try to answer, but more often I read a question and before answering I have a guilt trip because I'm answering to this question and left alone all the others and so I simply avoid this one too.

Most of all, the project is not aligned anymore to my interests, so it feels like a chore.

So, if you are reading this: think twice before making the same mistake. If you want a side project, pick one that is either small, or has a clear end to it.

I've never written a programming language. I have built a number of constructed languages[1] over the past few decades, which is an equally selfish (and thankless) task.

Less tangentially, I've worked for the past 8 years on building a HTML5 canvas library in Javascript[2]. I don't ever see people talking about JS libraries in terms of computer language design but I think, in a way, that is effectively what a (good) library is doing: building new, sometimes more developer-friendly, ways of conceptualising and creating programs that do stuff using novel APIs, syntax, state, etc.

There's a lot in the OP that resonates with my experience:

- "once you design a programming language you fall down a rabbit hole" - I've lost 8 years of my life to my project; but I've also gained 8 years of learning skills and insights that do translate to my everyday work as a frontend developer. For instance, almost all of my knowledge about accessibility comes from trying out many (many) different approaches to making the canvas element more accessible.

- "You keep adding new features that are really cool" - Guilty! My library doesn't need an integrated reaction-diffusion engine, but somehow it's acquired one along the way. Similar story with the reduced palette filter which, while a lot of fun to build and play with, remains too slow for use in production eg dithering live-stream video.

- "It also affects your communication with the rest of the programming community" - Because of the need for speedy calculations, I've developed an approach to programming Javascript which tends to avoid slower functionality. This sometimes leads to me writing code that could/should be written in a clearer, more concise and understandable way: 99% of JS code doesn't need to be super-fast, just fast enough!

[1] - for example, this one: https://gevey.rikweb.org.uk/

[2] - Scrawl-canvas - https://github.com/KaliedaRik/Scrawl-canvas

> your language needs an IDE, right?

If it does, it’s a bad language.

(IDEs can be beneficial for many languages. When those benefits become “needs”, that’s a sign that language is pushing complexity onto the editing tools, which has many bad consequences)

clojure has pretty good infrastructure for hosting new programming languages that can benefit from standard syntax, data structures and information model (immutability / abstract map reduce), foreign call conventions, macros, js/jvm targets. at hyperfiddle we are developing a multi-tier programming language specialized for server streamed user interfaces and hosting it with clojure lets us inherit a ton of free infrastructure, ecosystem, community and immediate commercial utility (which gets you serious use cases quickly - so important)

> GUI programming (your language needs an IDE, right?)

No language needs an IDE.

> I'm sure you've all seen the person, that guy who can't help himself from answering programming questions with an explanation how it's done in their own language.

Agree, this can be quite annoying, especially if you're looking for a practical solution to something.

> The person that discusses software optimisation techniques with regards how they implemented it in their compiler.

This on the other hand is often gold. I'd encourage the author and anyone else to share this knowledge whenever they feel the urge!

> The answer to the Infamous Question is always the same, and if the weasels ever ask it of you, snap back as quickly as possible, "Because I want to know how it works."

Jeff Duntemann

FWIW, I would assume this is the language: https://github.com/lokedhs/array

Of course, you can save yourself a ton of trouble by implementing your language on top of the GCC toolchain — the purpose for which it was invented.

Or you can write a translator which converts yourLang into existinglang, then interpret/compile the latter.

Or you can write an interpreter, which is generally less effort than a compiler.

All these are less ambitious undertakings than the full set of subtasks you enumerate.

Out of interest, does anyone have any book/article/resource recommendations for writing a programming language? This is something I would like to revisit, not to write a 'good' language, but to revisit all the concepts I was introduced to in college in practice rather than in theory.

No, write whatever the hell you feel like writing. Write whatever scratches your itch. End of conversation.

Please do. I have standardized set of solutions and I am lost if the environment does not conform.

Solution: make your own environment on top. https://github.com/timonoko/nokolis.py

don't write a programming language because i couldn't do it. also, don't host your own mail server, because i couldn't do that either. /s

don't listen to these naysayers, do it and learn. "Succeed" or "fail"; however you define these, there is great benefit to be gained.

being able to write a compiler, at whatever level of complexity, is immensely powerful and helpful. don't listen to the naysayers that the compilers class is "hard" -- no matter how far you get, it'll will pay dividends and help you solve more difficult problems more efficiently.

I think folks are misunderstanding the slight sarcasm in this article. The way writing a language will change your brain is a good thing.

Just wanted to say I've been working on mine for a year or so now, and most of this doesn't line up with my experience

Creating a programming language must be both one of the hardest and most unthankful jobs in the world.

You can probably adapt an existing small language to your needs if those are custom needs.

What about writing a DLS for a specific problem at hand?

This is a really really half-backed article or half-a__ed experience.. Sorry it's getting on my nerves.

Please, do not ever preach someone that if you cannot do something.

...but how will we solve new problems?