Martin Fowler's Domain-Specific Languages book is out | Hacker News Reader

Martin Fowler's Domain-Specific Languages book is out

martinfowler.com

63 pointspraeclarum21 comments

Perhaps someone who knows this material can advise us about something. I just looked through the linked ToC and am surprised that nearly everything is about DSL internals. Example:

Express statement sub-elements of a function call by putting them into a closure in an argument.

This is all easy stuff! The hard part is figuring out what a DSL should be. For all the hype about them, there are precious few examples of good ones. The successful ones are nearly always purely technical things like regular expressions that, while marvelous, do not even attempt to deliver on the promise of accessibility to domain experts. By far the majority are junk because they do a poor job of modeling the domain. This has little to do with closures or even parsers. To focus on such details is to take the easy way out (though I suppose it might sell more books to programmers infatuated with the idea of DSLs, who are more interested in, say, syntax-directed translation than in insurance or crop yields).

When asked what they thought of Elvis Costello, the Sex Pistols replied, "He wears glasses without any actual glass in them." Surely Martin Fowler has not written a Domain Specific Language book without any actual Domain in it?

I interned at a language engineering research lab in 2003-2005 and I heard the exact same sentiment about DSL's from most people working there. All the typical examples are techie languages (SQL, Rake, ...). I've almost never seen a non-techie DSL whereas that is ultimately the promise of the concept. It's sad the DSL story has barely changed in seven years.

I haven't read the DSL book, but if it's about parser/evaluator technicalities as you claim, one might as well (re)read SICP.

I noticed another thing about this book. The index has a dozen or so references to Lisp, but no mention of Forth. This is unfortunate. I bet there are more true DSLs in production in Forth systems than Lisp ones. They're scattered throughout the industrial world. Real people, i.e. with no knowledge of computing who are simply at work in their domain, use them to do things like calibrate lathes. Since the main thing we need if DSLs are to become more than design patterns squared (a technical scourge making things unnecessarily complicated for years to come) are real working examples, it's bad to ignore the richest source thereof. Of course this is one of the hard things about writing about software; where are you going to find such examples? On the other hand, I don't think people try very hard.

The DSL philosophy runs as deep in Forth as it does in Lisp. Consider this passage from Thinking Forth:

"Programming languages designed especially for particular applications such as robotics, inventory control, statistics, etc., are known as 'application-oriented languages'. FORTH is a programming environment for creating application-oriented languages. (That last sentence may be the most succinct description of FORTH that you'll find.) In fact, you shouldn't write any serious application in FORTH; as a language it's simply not powerful enough. What you should do is write your own languages in FORTH (lexicons) to model your understanding of the problem, in which you can elegantly describe its solution."

That's from 1984. Yeah, that's right: 19 freaking 84. Has there ever been a clearer description of the idea?

_Thinking Forth_ - discusses a bunch of ideas that didn't become popular until Fowler et. al invented them. (Incremental development, (re)"factoring", etc.)

A lot of those ideas came from cross-pollination between the Forth and Smalltalk communities, didn't they? Just like many others came from Lisp.

Please assure me you omitted quotes from "invented" in some kind of ironic seizure. :)

I'm not aware of any interaction between the Forth and Smalltalk communities, nor did I ever hear Forth referenced in oral histories along the Smalltalk->XP axis. It would be interesting if this were true. I bet your question reduces approximately to, "Was Ward Cunningham a Forth hacker?"

Moore was influenced by both APL and (especially) Lisp, which he studied with McCarthy. But Forth predates Smalltalk, and the two moved in very different circles. Still do I guess.

While I'm at it, I've been meaning to write the following down somewhere. Out of the great small, regular languages (Lisp, Smalltalk, APL, Forth, is that it?) Forth stands out as the only one that originated in working hackerdom and not a research environment. Forth is the incarnation of that design style in the land of the Von Neumann architecture, doubtless because Moore was a working programmer all those years, just trying to get things done. The remarkability of this fact has yet to be appreciated. Consider Lisp by contrast: it began existence far away from the machine and has been trying to find its way back ever since (or bring the machine up to it). With Lisp, bottom-up programming nonetheless starts pretty high up; Forth starts at rock bottom. That's important, and I wish more of us (myself included) had a better grasp of it. One could do worse than to make it the introduction to computing, or at least teach it alongside Scheme/Python/etc. I think it would be very exciting to young hackers to begin at the beginning in this way.

I think I read about Forth influencing Smalltalkers on the c2 wiki somewhere - IIRC, it had more to do with the problem solving approach than the language itself. I could be wrong, though.

Edit: Found one reference - "It must have been ten years ago that I tried to explain the essence of Smalltalk to my brother, a Forth programmer. We found common ground by programming -- one day in Forth, another in Smalltalk. In Forth we wrote the essential Smalltalk: object identity and method dispatch; in Smalltalk we wrote the essential Forth: dictionary, words and builds-does. We both showed off our respective environments with great skill, pride and confidence. Oh, if only all programming could be so educational. -- WardCunningham". (http://c2.com/cgi/wiki?ForthInSmalltalk) So yes, Ward Cunningham knew Forth, for one.

As for other "great small, regular languages", there's also ML, Prolog, and SNOBOL. I haven't used SNOBOL for anything, but it AFAICT it introduced pattern matching, so it's been pretty influential. There's probably more good candidates in HOPL I and II. Maybe also awk.

(And yes - the original version of the "invented" sentence was positively unctuous. ;) )

I would say the same for Prolog. Since doing general programming is so different in Prolog, it has been used mainly for the implementation of DSLs. It is really nice to read papers where a apparently complicated DSL is implemented in < 100 lines of code.

Or you could just read Chapter 4 of SICP: http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-25.html..., the first edition of which was published 26 years ago.

This is an excellent suggestion. Other suggestions: read the wikipedia articles on Simple LR and General LR parsers; google "python descriptor" and read the top results.

A parser does not a DSL make. I find this advice, and the advice of the parent, dangerous, as it suggests writing a DSL is just a, relatively simple, technical challenge.

It is like advising the Dragon book and someone replying 'meh, just read SICP'. As in that case, SICP or articles on LR parsers do not even remotely stack up to the Fowler DSL book combined with the ANTLR reference.

Two things: (1) classifying Chapter 4 of SICP as a chapter about parsing is just plain wrong and (2) Fowler's DSL book just came out; by definition you simply can't place it in the same class as the Dragon book.

I think if you actually read Chapter 4 you would have a much different opinion. There's nothing "simple" about it. If you actually grok Chapter 4 then you grok Lisp. After that, it becomes apparent that the phrase "Domain Specific Language" is really just unnecessary terminology.

  classifying Chapter 4 of SICP as a chapter about parsing
  is just plain wrong

I classified my parent as suggesting writing a DSL is mostly writing a parser. I classified your post as saying writing a DSL is mostly a technical challenge, because there are many issues about writing a DSL that SICP doesn't touch upon (or at least, that's mostly what you will get from it on a first reading. See the final paragraph of this reply).

  Fowler's DSL book just came out; by definition you simply 
  can't place it in the same class as the Dragon book.

I didn't place it in 'the same class': I suggested Fowler's book plus the ANTLR reference have many interesting, relevant, even necessary, things to say about the subject of writing DSL's that aren't in SICP (but again: see the final paragraph of this reply), like the Dragon book has many interesting, relevant and necessary things to say about compilers that aren't in SICP.

  After that, it becomes apparent that the phrase "Domain
  Specific Language" is really just unnecessary terminology.

You didn't originally get that just from chapter 4 of SICP. You have much, much more knowledge of all kinds of things relevant to writing DSL's that you didn't learn from chapter 4 of SICP. That it may all be implied in that chapter does not mean it is a useful resource for someone touching the subject of DSL's for the first time, because they won't grasp the implications. Implications may be clear in hindsight, when you apply all your other knowledge to that chapter, but that doesn't mean they were even remotely clear to you in the first place.

  If you actually grok Chapter 4 then you grok Lisp

And you won't grok that chapter until long after you have acquired a vast array of secondary, additional, related knowledge on the subjects the chapter touches upon.

SICP is like a deep philosophical work: you need to study philosophy and read a lot of primary and secondary works to truly understand it's references and implications. To most people, it is not useful as an initial guide to writing DSL's.

You didn't originally get that just from chapter 4 of SICP. You have much, much more knowledge of all kinds of things relevant to writing DSL's that you didn't learn from chapter 4 of SICP. That it may all be implied in that chapter does not mean it is a useful resource for someone touching the subject of DSL's for the first time, because they won't grasp the implications. Implications may be clear in hindsight, when you apply all your other knowledge to that chapter, but that doesn't mean they were even remotely clear to you in the first place.

This is a fair point and worth talking about. When I read SICP I already understood Lisp macros, which is a major conceptual leap. So I will admit that I cannot provide a datapoint myself to support the idea that Chapter 4 is stand alone. However, the text itself was used in MIT's first computer science course for many years and they did teach through Chapter 4. Many other schools have used SICP in the past as well (http://mitpress.mit.edu/sicp/adopt-list.html). I think the fact that college freshman are capable of grasping it in a semester provides some evidence against your claim that you need "much, much more knowledge of all kinds of things". Hell, even spending a year to learn it would be fantastic.

Look, I'm with you on the fact that SICP is hard. But let's just reflect on the fact that Chapters 4 and 5 in a book written 26 (or 14) years ago give you a strong foundation in programming languages and compilers. There's tons of stuff you can do with this foundation, including writing DSLs. It's a pet peeve of mine that we seem to be constantly rehashing various ideas in SICP, except that we need to write thousand page books on how to express these ideas in languages like Java or C#. Based on a quick glance of the book draft, this bloat is immediately apparent! For example, the distinction between internel and external DSLs? Completely unnecessary. With an understanding of Lisp the idea of an internal DSL disappears; it's just a natural part of the language. And when you think about it, the idea of external DSLs isn't useful either. If the external DSL is expressive enough, it's just like an internal DSL; simply an extension of the language you implemented it in. If not, well then you're parsing am external file format and not a language.

So yes, to reiterate, teaching SICP is hard. But we'd be better off just biting the bullet to avoid all the bloat and repetition that results from not teaching it.

  I think the fact that college freshman are capable of
  grasping it in a semester provides some evidence against
  your claim that you need "much, much more knowledge of all
  kinds of things".

I would say they probably grasp it as much as I grasped quantum mechanics after the first course: I was able to go through the motions and explain the concepts involved, but that doesn't mean I actually understood it. I would claim that at some later point, after more courses, labwork and discussions on the subject, I did understand certain parts of it. Much of the additional knowledge and principles I learned are already implied in decent basic texts on the subject, but I simply wasn't in any position to 'expand' those implications.

  There's tons of stuff you can do with this foundation,
  including writing DSLs.

However, writing your first DSL on that foundation requires you to reinvent many wheels, some of which you will reinvent in a square way that first time. Knowing how to write a programming language in Lisp doesn't tell you how to write DSL's, like having a Ph.D. in English means you know how to write a novel, without you actually being capable of writing an good novel. They would still benefit from reading books on 'writing novels' by people that have practical knowledge in writing actually good novels, even if those books were written by authors whose genre they dislike and the style of whom's examples they dislike.

  the distinction between internal and external DSLs?
  Completely unnecessary.

If everyone would program in Lisp: yes. However, people write DSL's in different languages, for all kinds of reasons you may or may not agree with. That means books on writing a DSL in other languages than Lisp simply have their place. And may also be valuable for those writing DSL's in Lisp, for the reasons mentioned above.

I found pg's On Lisp to be more enlightening on this subject. Available for free at http://www.paulgraham.com/onlisp.html.

I bought this book as a "Rough Cut" a few months ago. If you want a very high-level introduction to the field of DSLs this book may be for you but don't expect to be able to do much useful after reading it. It really is just the 50,000 ft view.

I was disappointed because I wanted to get my hands dirty and so I got The Definitive ANTLR Reference a day or two later. While it has nothing like the breadth of Fowler's DSLs it has enough depth to be very useful.

http://www.pragprog.com/titles/tpantlr/the-definitive-antlr-...

As a newcomer to writing DSL's, I found the book very valuable as an introduction to all the intricacies that can be involved in writing a useful, nicely integrating, DSL. The abstraction of the 'semantic model', which Fowler advocates strongly, has proven indispensable, allowing me to easily build GUI-based DSL editor for an initially completely text-based DSL, without changing a single bit in the code executing the parsed DSL.

Incidentally, this book and the definitive ANTLR reference are exactly the two books that appeared on my radar when searching for literature in DSL's.

on amazon: http://www.amazon.com/Domain-Specific-Languages-Addison-Wesl...

The link has changed?

http://martinfowler.com/dslwip-gone.html

"Now that my book on Domain Specific Languages is published, I have taken down the online drafts from my web site"

Needs web design

Martin Fowler seems so 2008.

To think I believed him that TDD was a good idea... It seems so far in the past.

I vote MF take Zed Shaw up on his ring match offer. At least ZS has an honest job now, soaking up government-guarantee student loans while MF seems to still be rolling ignorant corporations with his "experience"...

Reply on news.ycombinator.com