Diving into Domain-Specific Languages: A Practical Guide for Developers

I've created a couple of things that can be plausibly called interesting DSLs, in the sense of having some grotesquely non-standard execution model relative to the host language, and having language-like recursive grammar constructs, but so far I've managed to keep them within YAML and/or JSON and thus didn't have to create a new language.

(Before someone pops up with "ick! YAML!", consider your options; do you want to encounter your fifth or sixth YAML format, or you want to encounter a brand new bespoke language for whatever it is I solved this way? At least you know how YAML is broken; Jerf's Bespoke Language is a minefield of "interesting" of unknown size and density, and I'm just talking in grammar... we still haven't even gotten to have it actually functions and you've already blown more limbs off than you realized you had.)

Going back even farther, the "interpreter" pattern in the GoF is extraordinarily powerful, but in 2024 I'd update it to if at all possible be written in the form of some existing serialization mechanism that then serializes straight into your AST(-equivalent) and starts off to the races from there. If you have to write a real grammar, well, there may be times that is unavoidable, but if you can avoid it, you can cut much more directly to the "meat" of your task without having to deal with bringing up a complete grammar first.

I then redid it as an external DSL, which has been successful, but a vast amount of work. A more limited environment has a lot of advantages for my users.

https://github.com/matry/tree-sitter-matry

I can name at least 3 ML DSLs that you've heard of.

The huge number of Flask-like stacks makes me think that’s a significant, use case. Double true if the host stack supports live updates of running applications.

You might also look up the iMatix DSL’s that had high-level constructs that compiled to low-level code.

We measure success along a few indicators:

- Ease of deployment of an optimization model to production

- Training time required before a new employee (with a background in Supply Chain, but no software development experience) is proficient enough to contribute new code and edit existing code.

- Cost of running all the execution infrastructure

- Frequency of defects caused by running out of memory

- Frequency of package version upgrades that require manual intervention

- Frequency of unsuccessful attempts at auto-completion and other Language Server Protocol interactions

1: https://docs.lokad.com/ 2: https://www.youtube.com/watch?v=_gmMJwjg0Pk

One was for a visual novel video game, where the designers had relatively detailed specs. They had been manually translating those to code, but that was tedious and pretty far outside their expertise. I was contracted to translate those specs into code. I ended up mostly writing a script to turn their specification language into a more structured representation, then translate that into code, with a bit of back & forth to make their language more precise. From everything I heard this was a big success, my script saved them a lot of hours, and the game was pretty successful by the standards of indie games.

The other...is not exactly a DSL, but I don't know what to call it. We had researchers who were writing a lot of numeric code to try and find signals in data. This code often worked, but was slow and expensive to run at large scale, in part because it would have to recalculate intermediate values over and over again. So we wrote a "DSL" that had almost exactly the same syntax as the code they were writing, but was lazy and created a DAG rather than immediately executing, and eventually used the DAG to execute much more efficiently.