Hacker News Reader

Comment by taylorallred | Hacker News Reader

Languages that encourage making DSLs are a two-edged sword. On the one hand, you get to make a language that is more clear and fine-tuned to your use-case. On the other, you have an ad-hoc language with no support that you have to maintain along with the documentation (considering that you can't expect anyone else to know the DSL ahead of time). As I've gotten older, I've determined that well-designed APIs in a well-known language are a better alternative to DSLs.

View on HN

I’m right there with you. Modern languages have such a plethora of tooling available that DSLs and any API approaching a DSL have this massive gap unless they really embrace whatever patterns the host language’s tooling enables.

Not long ago, i had to work with a coworker’s mini language and function runner engine, which was basically a mini programming language. Except without a debugger or type checker or stack traces or any of the other million niceties we’d have had if we just used the host language to execute things ‘natively.’

That said, while the level of tooling for big languages goes up, the bar for creating yesteryear’s tooling is going down, with all the LSP tooling we have now, for example. Maybe someday we’ll get languages with tools where libraries have nice tooling without crazy dev effort, and then we’ll change our tune on DSLs.

I like the idea of DSL, but in real projects, I hate almost every DSL introduced by coworkers.

Because they are really really difficult to get right almost as difficult as building a new language. But when they work well they can be a real boost.

I've come to hate DSLs. With rare exception, almost every DSL has become a brittle subset of the parent language.

Sometimes you want a specific limited subset of the parent language, especially when you don't want to expose a huge possibility space to non technical users. Exposing only a limited subset can also be helpful to sandbox for security reasons.

Even DSLs of huge and successful projects are more often than not bad. As a rule if you think that you can create a better query language than SQL, please spare us. JQL, Elastic Query DSL, KQL... Don't.

It has been interesting to watch the Clojure community’s take on DSLs over time. Clojure is a Lisp, and Lisps are notorious for inventing DSLs, particularly after PG’s various papers promoted that as a superpower of sorts. Clojure has gone the other direction and, while it supports macros, it generally discourages their use unless you’ve tried everything else first. But it still encourages DSLs built from the native Clojure data structures. So, for instance Hiccup for HTML generation.

The discouragement of macros I think is a knee-jerk reaction. Build around data structures and functions first, and then use the thinnest macros around that can be quite maintainable.

I've been thinking about this a lot. DSLs solve an important problem, but they do it in the wrong direction.

It's impossible to write objective fact. Everything we write is subjected to the context it is expressed in, including the grammar that we use to write it. A DSL accommodates this by letting you make a new grammar on the fly. The trouble is, this doesn't help us get out of the context we are writing in already: it only lets us enter a new one that is nested inside.

So what if we could actually get out? What if we could write from outside of the context we are writing? That's the idea I'm working on. I think it's possible, but it's such an abstract idea that it's tricky to get a handle on.

An API is just as much a DSL.

Kind of, except that a non-DSL API doesn't create any new syntax. Which means that you get to keep all sorts of quality-of-life tools like syntax highlighting and correctness checking in the editor, autoformatting, possibly some amount of linting, etc.

A few years ago I revisited Racket after a long hiatus, and that was maybe the biggest thing I noticed. I really don't like syntax macros as much as I did back in the day. Once I decide to use `define-syntax` I've then got to decide whether I also want to wade into dealing with also implementing a syntax colorer or an indenter as part of my #lang. And if I do decide to do that, then I've got a bunch more work, and am also probably committing to working in DrRacket (otherwise I'd rather stay in emacs) because that's the only editor that supports those features, and it just turns into a whole quagmire.

And it's arguably even worse outside of Racket, where I might have to implement a whole language server and editor plugin to accomplish the same.

Versus, if I can do what I need to do with a reasonably tidy API, then I can get those quality of life things without all the extra maintenance burden.

None of this was a big deal 20 years ago. My expectations were different back then, because I hadn't been spoiled by things like the language server protocol and everyone (finally) agreeing that autoformatting is a Good Thing.

Even without the new spiffs, I still don't see the point of DSLs. From where I sit, I see exactly zero problems where I think that new syntax is what I need to be able to write a solution.

Regex

Fair point, though I learned regexes a long time ago, so that they weren't on my "problems that I can solve with new syntax" list, but instead on the "syntax that's already there" list.

TBH, I feel like regexes would be much easier to understand in a more literate form with standard syntax. Something like:

  user_part = re.repeat(re.alnum | re.chars(".-_+"))
  domain_segment = re.repeat(re.alnum)
  domain = re.list(domain_segment,separator=".",minimum=2)
  email_address = user_part + "@" + domain

Where, in a real program, `domain` would be defined in a "standard library of constructions" that you can just import and re-use in more complicated regexes.

Something like this can be implemented in any language with operator overloading, no DSL required. Without operator overloading, the syntax would be a bit more awkward, but still nicer than the current regexp madness.

I don't quite understand where regex gets its reputation from. I think that once you remember the meaning of the operators, it's not too bad. (And the concise syntax is actually very helpful.)

I get that the meaning of the operators is not clear unless you're already familiar with regex, but neither is the meaning of !, ?, %, &, |, ^, ~, &&, ||, <<, >>, *, //, &, ++ (prefix), ++ (postfix), and so on. You learn these because you need them once, and then they're burned into your mind forever. Regex was similar for me.

Also, regexps are just that: regular. You can mostly read them from left to right decoding each symbol at a time.

I think regex gets some of its hate from people writing painfully complex matchers. 99% of my day to day regex use is simpler string searches on the command line or in my editor. I’m really happy I took the time to learn the syntax (spent about a week on it around 15 years ago) because now it doesn’t get in my way.

The limitation with regex is that's it's context free, whereas a regular grammar is simpler and more powerful IMO.

Regex syntax helps you understand what a regex does, not necessarily why it does it.

You can't decompose it into parts, you can't give those parts human-friendly names, you can't re-use parts in other regexes, you can't (easily) write functions that return or manipulate regexes (like that "list with separator" function shown above).

recommend you check out raku Grammars … https://docs.raku.org/language/grammars

Or in Red's case, Parse - https://www.red-lang.org/2013/11/041-introducing-parse.html

There’s really nothing about this that couldn’t be expressed just as clearly in a more general API though, even in C, IMO. Building it into the syntax is a little weird to me.

Or in Rebol/Red it's Parse, a dialect (DSL) that comes with the language - https://www.rebol.com/docs/core23/rebolcore-15.html

I used to feel that way. I’m still not a convert, but now I’ve seen a lot more complexity papered over by a nice DSL.

Standard math syntax is a DSL. I understand math a lot more quickly than I understand the same thing written in 20 lines of code.

I think the language we use to express ourselves influence the quality of the product. If your language encapsulates complexity, then you can build more complicated things.

I’m not arguing in favor of specific (“pointless”) DSLs, but there’s a nice paper about making a video editing language in Racket [1] that makes a DSL seem pretty convincing.

[1]: https://www2.ccs.neu.edu/racket/pubs/icfp17-acf.pdf

The protocol buffer / grpc definition language is another great example of where a DSL can shine. Especially if you compare it to efforts to accomplish basically the same task using pre-existing languages, such as OpenAPI (JSON) and WCF (XML).

Captain Pedant here, but OpenAPI also accepts yaml which suffers from a ton less { and "

gRPC and I are not friends and I've thankfully never needed to interact with WCF

Except in homoiconic languages, like Rebol/Red/Lisp, there is no (need for a) new syntax.

Lisps don't support OO, concurrency like Go, or type checking but you can write a DSLs for them. Not useful? How about a rules engine?

Lisps support OO like you wouldn’t believe: https://lispcookbook.github.io/cl-cookbook/clos.html

You could write DSLs for them, but you could just as easily do it within the existing syntax. Which is exactly how all the popular lisp implementations of OO, concurrency and type checking work in practice.

Many of them do use macros. But that's not about creating a special language; that's about moving expensive computations and checks that can be done statically to compile time where they belong.

Any markup language is a DSL. Including HTML.

Netlists.

Makefiles.

And so on.

You really don't see the value of DSLs?

Conversely, Ruby is often seen as facilitating DSL's, but none of it changes syntax, because the syntax as-is is flexible enough that redefining methods is sufficient. But everything still abides by the same syntactical rules.

The more orthogonal or flexible the language is, the less there tends to be a distinction between redefining syntactical elements and defining functions or methods.

> Kind of, except that a non-DSL API doesn't create any new syntax.

Internal DSLs are always (by definition) valid code in the host language; external DSL, where parsing and interpretation or compilation to executable code for raw text code in the DSL are implemented in the host language, are all new syntax, but “languages that encourage making DSL” are usually ones that are favored for internal, not external, DSLs.

I think that "internal DSL" is what the grandparent poster meant by API, so I just went with that.

I think the internal/external terminology might be kind of outdated, anyway? I think this might be the first time I've encountered it in the wild in over 10 years.

> I think that "internal DSL" is what the grandparent poster meant by API, so I just went with that.

When people talk about REBOL/Red (or Ruby or Lisps) encouraging making DSLs, they are referring to internal DSLs. In Ruby, these are just APIs consisting of normal objects and methods, in Lisps they often involve macro calls, which may or may not correspond to what people mean by an API, and in REBOL/Red the design of the language is such that “normal” functions can do things that would take macros in Lisp.

No, an API uses existing rules, but a DSL uses its own ad hoc rules.

GP is right. Don't make DSLs, make APIs, which are:

* More composable

* More reusable

* More simple to reason about

* More natively supported

* More portable

* More readable

* More maintainable

There are plenty of places where it makes more sense to use DSLs (or where they’re flat-out required). SQL and regex both spring to mind. Pretty much any HTML templating language is simpler to use than concatenation a bunch of strings together. JSX is usually easier read than directly calling React functions directly. A line of a shell script can be much nicer (and more portable) than 20 lines of a more general purpose language.

Those are things that spring to mind that I think are unequivocally DSLs, but if you’re willing to consider markup languages as DSLs, the list could get a lot longer.

Most people think of DSLs as a language you can create within a language. All the things you named are either entire languages themselves like HTML, SQL, and the shell family, or formal extensions of existing languages like JSX. A DSL would be something you could create inside JS tagged literals or Ruby.

I don't think of DSLs as strictly being embedded in another programming language. The term I would use for that is "eDSL", short for Embedded Domain Specific Language. See: https://wiki.haskell.org/Embedded_domain_specific_language

Martin Fowler on DSLs, including internal and external DSLs:

https://martinfowler.com/books/dsl.html

https://martinfowler.com/dsl.html

Also see:

https://en.m.wikipedia.org/wiki/Domain-specific_language

including the References section.

I've seen it. Sometimes a DSL is more readable than trying to shoehorn control flow into method calls (".then().catch()..."). Or see C#'s LINQ.

> Or see C#'s LINQ.

This might be one of the rare times it's worth it. The C# team alread has the experience and tooling to maintain a language. Maintaining a DSL might be a reasonable choice for them.

It's rarely a good idea for app or library devs to make a similar decision.

Yes, this reminds me of meta-programming in languages like Python, it's useful if you want to create a framework like Django, but if you work on products, chances are good you should not use it.

An API creates its own ad hoc rules too. They just don't change the grammar.

In languages where the grammar is sufficiently flexible, the distinction all but disappears, but even in languages where the grammar is rigid and API's stand out like a sore thumb, the API itself still creates a new rule-set that you need to learn.

You can choose to not call that a new language all you want, but the cognitive load is still there.

An API (as people usually mean by it, anyway…) is really just a simple grammar for the creation and manipulation of objects in the system. Those objects still have types and properties just like any other object, but they tend to be implicit and unreliable. A good DSL is just a more abstract and implicitly verb-oriented API. For example, Lua DSLs can resemble something almost like a sort of typed or tagged DDL. Translating that into an “API” just adds more clumsy legwork to achieve the same outcome.

Agreed - I was tempted to make the same claim that it is still a grammar, but decided against it as I suspected it'd get in the way of making the point. You're 100% right you can write a grammar for an API as well, or can consider the API definition it self as creating a grammar.

No, an API is a contract. Just like an object or a class or a structure is not a DSL. None of these modify the syntax or keywords of the language used to implement them, and only objects and classes actually have the ability to override operators.

Just grouping things together in a particular order and giving the group a name does not make a DSL.

The API creates a language just the same modifying syntax, by introducing new words with new meaning.

The point is that however you want to group it, the effect is the same: You introduce a whole new vocabulary that you need to learn the rules of.

A DSL is expected to add or break language rules. An API is not.

An API can be so complex as to be thought of as a DSL but that makes it bad.

An API is an interface, metaprogramming is a technique for building such interfaces by treating code as data. They aren't interchangeable concepts.

Metaprogramming comes in handy when ordinary programming results in a lot of complex, hence unmaintainable, code or can't achieve something due to how the language is implemented. It can vastly simplify how you express application logic, e.g. when interfacing with foreign programming languages such as SQL or generating code from data.

Any sane metaprogramming regime allows you to do something like macro expansion, i.e. look at the code the metaprogramming is compiled into in the current context.

this is a very important lesson. I remember the frustration of dealing with the Gradle and whatever was the name of the DSL used by Fastlane. Gradle especially kept changing so documentation was never really useful and there was zero help from the tooling

I've run the same frustration road.

And at least the last time I used it in anger (which was, granted, perhaps double digit years ago), the docs started out with and loudly proclaimed "here's how you write your own tasks!" Wait, what? I want to have a build system do at least the 80% case with minimal/zero work; why am I jumping right in to my bespoke needs?

Not sure if this was just bad doc, or the tool actually didn't do the 80% very well then or what, but it's stuck with me.

> and there was zero help from the tooling

That's mostly my experience with ruby, too: you can type anything, don't worry about it!

I wanted to mention that Gradle since I think about version 7 supports mostly static typing via Kotlin but regrettably it's lipstick on a pig since almost inevitably one needs to interact with the Groovy side of the house and then all bets are off

I heard that in Lisp every program becomes a DSL?

I wouldn't say that. It's just its trivial to make DSLs in languages like the various lisps (common lisp, racket, chez scheme...etc).

That's usually what people complain about (besides the parenthesis) in that it's easy to not be very disciplined.

Lisp is homoiconic, so there isn't really a distinction between programs and data. For example, a snippet of code like a for-loop iterating through a list is also a list that can be inspected and modified. Or something along those lines (there's an XKCD comic that captures the spirit where the person says "it's all CARs" as in lisp you can build everything from CAR, CDR, and CONS I think). You'll have to dig into that on your own. The terms are historically relevant, but seem antiquated now. I've never really understood macros (compile or runtime ones) all that well though, so hopefully someone else in the comics can clarify my mumbo jumbo.

https://www.explainxkcd.com/wiki/index.php/224:_Lisp

Reply on news.ycombinator.com