Hacker News Reader

The problem with writing a DSL for every problem (and every programmer writing different DSLs) is that you end up with all these different languages with different vocabularies and different semantics. It's like a million different variations of Esperanto.

Using a programming language is like speaking in common tongue to every other programmer who knows that language. And most languages are so similar that most programmers can communicate even between languages.

Having code that looks the same, regardless of what it's actually doing, allows everybody to be able to read it.

>Using a programming language is like speaking in common tongue to every other programmer who knows that language

As much as I love DSLs, I Couldn't agree more.

>The problem with writing a DSL for every problem (and every programmer writing different DSLs) is that you end up with all these different languages

When writing in a DSL of my design, it is amazing. It feels like the program models exactly how I am thinking about it.

When modifying/maintaining a DSL made by someone other than me, often this can be horrifying, it feels as though this models my thoughts so poorly (and it does, it was designed to model their thoughts).

"Welcome to our team, here's the spec for our project, and here's the spec for our DSL. See you in a month."

Actually this is how it was when I started at my current position.

We are doing web-testing in clojure, and in past year our little framework evolved into a DSL of a sorts.

On one hand it is great, we a have consturct like (with-client [name connection & body]), that behind the scenes sets up a virtual machine, registers it as a client of our web-service and destroys it after commands in body are done. I can't imagine we would be doing setup and teardown by hand.

On the other hand it took me a month to get into it (two weeks learning the language, two weeks messing around with the framework) ... and there are still lots of things only our lead-tester knows how to fix, after our devs decide to change something we were relying on for testing.

>two weeks learning the language

I am curious, did you already know the JVM? When I learned Clojure, I was also learning the JVM, and for me the JVM was much harder than Clojure. So I would say it took me 6 months to learn Clojure, but much of that time was the time it took to learn about the JVM and all the related Java weirdness.

Either way, 2 weeks is very impressive.

JVM? I knew Java, but most of what I did in java was using autocomplete in Netbeans and cursing Maven, when it didn't work.

What really helped me that, we had "programing paradigm" course in college, where I learned haskell for 3 weeks, what was enough that anonymous functions, map, reduce and using lists for everything didn't feel new.

"and here is the spec for our in-house framework".

"and here is the 1M LOC codebase, which was carefully crafted to avoid using any of the complicated frameworks; all you gotta know is the C standard library".

Spec? For an in-house framework? Luxury!

Did I say Spec? I meant the doxygen output of hundreds of classes with sparse one-line doccumments.

Plus one outdated sequence diagram.

One-line comment? What are you, a writer-librarian?

It can be learned. Observe:

       //! \brief Transport diagnostic wrapper
       class TransportDiagnosticWrapper {

(Actually, meaningful one-line comments are a lyric discipline of their own, related to haikus.)

You joke, but a tool I have seen lauded for being really "smart" at automatically generating doxygen comments does exactly that. I guess it at least saves the time of a human doing it by hand . . .

If you start a sentence with "Welcome to our team" and end it with "See you in a month." whatever you put in between, your company is doomed, and your team is highly dysfunctional.

Programmers already know the DSL for every problem: English (or their native tongue). If they're writing programs for a specific domain without knowing the language of the domain, they have problems that won't be solved by using programming terms they already know.

How many people have learned what MVC means from Rails? Or the term "routing" in the context of a webapp?

One caveat I'll give: the DSL must be well-conceived. Writing a DSL that is dissimilar to both the underlying language and the domain vernacular is obviously horrible.

I couldn't agree more. As much as I like languages that facilitate DSLs, this is an unrealistic view of software development.

To add to what you wrote, you have to also consider that all developers in one team have to agree on a common DSL beforehand, which is an investment of time in itself, and the fact that if there is an unusually big change in requirements mid-development (which, let's be honest, is not a rare event), you might have to go back and change your "pyramid of DSLs" to accommodate it, maintain compatibility and update all your colleagues on the new languages.

How is this any different than a team building a set of APIs?

I see APIs as a less extreme version of DSLs (in that they share a common language, while building a DSL frees you from almost any restriction), but I honestly don't have enough experience building APIs or DSLs to give you a satisfying answer. Sorry.

The syntax required to use the API will be instantly accessible to new developers. Many programming languages provide semantic guidelines for APIs as well. So knowing the language an API is programmed in gives you a huge advantage in beginning to understand it.

With a DSL, all that goes out the window. The syntax of the language is potentially unknown to you. The semantics are an even deeper mystery. You're in terra incognito, trying to understand the mental process of the developer of the DSL before you can even understand what the code written in the DSL intends to do.

The primary goal of programming languages is not to describe how to perform a complex task to a computer. It is to describe how to perform a complex task to other humans in a form that computer can also execute.

I'm not sure I buy the whole syntax argument. I programmed in Forth for a while and it seemed like it was much easier to build up an understandable vocabulary than some of the C++ / Smalltalk class hierarchies I have seen. REBOL has dialects that work fairly well. I cannot help but think that DSLs are refinement of APIs the same way C is a refinement of Assembler.

When I talk to someone about what happens in a system it seems like a DSL flows from that conversation.

"It is to describe how to perform a complex task to other humans in a form that computer can also execute."

I think a DSL does that much better than a set of API calls, particularly when you involve non-developers. That is the point of a DSL.

There's very little difference between a DSL and an API. You already know dozens of APIs. Why not DSLs?

That has been bugging a long time. Exactly what is the difference?

And statements like (paraphrasing the quote in the article)

> You shouldn't build an app in Forth. You should build a language to model your problem domain in forth and use it to build the app.

They just doesn't make sense to me. Isn't that the very definition of how to program?

For a lot of languages, I agree, because there's no way to create a DSL as part of the language itself. Think of the difference as one of direction; are you fitting the domain into the language, or extending the language to fit the domain? Most languages are either general-purpose (C, Java) or their domain is fixed upon creation (SQL, Mathematica).

In a Lisp, you can use macros to accomplish things that go way beyond defining a domain-specific library / API. A typical library is restricted to the capabilities of the language, but a DSL in Lisp can extend the syntax itself.

Want to alter the reader so you have a way to express normal code as remote computations? Go for it. Feel like adding new math operators to use matrices instead of scalars? Knock yourself out.

Common Lisp itself did exactly this when object-oriented programming came along; using nothing but macros and closures, the Common Lisp Object System (CLOS) was built.

The difference between DSL and API is the difference between language and library. What is that difference? Has it ever been successfully defined?

Abstraction layers are great... if you don't end up spending 90% of your time climbing the stairs that connect them!

Take SQL, a nice DSL for working with relational databases. Yet people ended up writing ORMs (quite ugly creatures imho - something between an elevator connecting two layers, and as in real life elevators are a mess when too many people want to use them at the same time, and an intermediate pseudo-level, but they are useful) because the pain of climbing up and down the ladder connecting the "relational level" with the "OO layer". "Language layers" or DSLs are a powerful but dangerous abstraction because when they start using them people can't stop, they just make more and more layers like crazy! (The only other abstraction so powerful and so dangerous I can think of are objects - it's amazing how many got them completely wrong so many times in languages like C++ or Java... besides Ruby and Smalltalk I can't think of anything that got them "mostly right"... and Ruby seems to be the most DSL-able language in use too, though this is why I'd take Python over it any day - smart programmers tend to make everything like a freakin DSL in Ruby and and your mind ends up running up and down the ladders until you go insane when reading code).

The other problem with SQL is the "standard problem": you have so may of them to choose for that when you need to support several databases, the additional layer of complexity and black magic of an ORM can be worth it.

I can't speak for everyone, but the reason I choose to use an ORM over writing all my SQL by hand is time and maintenance cost. I can write one SQL query to perform CRUD operations for a model almost as quickly as I could use a ORM. The problem is that it produces 10 times as much code to read and interprete. If I write the SQL code for another model, 90% of that new code is exactly the same as the last. I am bleeding duplication everywhere.

What is completely wrong about the way C++ does "objects?"

I've recently been digging hard-core into C++ after making such disparaging comments for years and have been rather surprised to see that it's not as terrible as I once thought -- It's actually quite good.

I agree however with your point about dangerous abstractions. DSLs when implemented with careful, thoughtful restraint are rather powerful. SQL being a really good example. There's nothing stopping anyone from writing their database queries by directly sifting through and filtering datums from the domain using the primitive functions and operations of their programming language of choice... but it's clear that SQL is much more powerful and lets us express such problems in a succinct way.

But just like programming languages, once introduced, a DSL is likely to be misunderstood and abused. The problem I believe lies not with the technique but with the users. A double-edged sword is a fine tool but will likely maim an inexperienced user. Which will probably lead them to writing DoubleEdgeSwordWrapperMachineGun classes and then someone else to write a DSL on top of that...

Your example is actually a pet peeve of mine. SQL is not at all a nice DSL for working with relational databases programmatically. When you think of it as an API, it exposes one method that takes one gigantic string parameter. API design doesn't get much worse than that.

A sane relational database API would let you compose and extend arbitrary queries, which has profound implications for the way you can write your program and abstract away repeated patterns.

[Arel](https://github.com/nkallen/arel) has the right idea in terms of API, but a poor implementation (lots of things are not truly composable where they should be). [ScalaQuery](http://www.scalaquery.org/) purports to have the right idea, but I haven't tried it, and it's very Scala-centric.

I would add Lua to the latter group (a language used as a tool to build the language you need).

Not NEARLY as cool of an implementation as implementing TCP using ASCII art, but there's a command-line option parser that parses the help text to determine the command line options. [1] For example:

    -- scale.lua
      require 'lapp'
      local args = lapp [[
      Does some calculations
        -o,--offset (default 0.0)  Offset to add to scaled number
        -s,--scale  (number)  Scaling factor
         <number> (number )  Number to be scaled
      ]]

      print(args.offset + args.scale * args.number)

[1] http://lua-users.org/wiki/LappFramework

That can be done in basically any dynamic language: see http://docopt.org/

It can even be done in static languages with an intermediate compilation step.

That's a pretty cool project. Sure you CAN do it in any language (as another commenter points out, you can do it in C too). Anything that's turing-complete can do it. You could also use C to parse ASCII art and have that create a TCP implementation. It's not the natural way you'd do it in that language, though.

lapp has been around for four years; docopt is (looking at git commits) less than a year old. It strikes me as a natural way to do things in Lua, which is why (I suspect) it was done in Lua earlier.

Threre's also LPeg [1], which I find much nicer than lex and yacc (or flex and bison) for writing parsers. I just finished up a quick hack to pull out a single file from a git repository into its own repository, and the parsing of the "git fast-export" output is 70 lines of Lua/LPeg. It may sound like quite a bit, but it's a near dump of the BNF you find on the man page [2]. From there, it was simple enough to grovel through the in-memory representation of the git repository and output only what I needed.

[1] http://www.inf.puc-rio.br/~roberto/lpeg/ [3]

[2] http://kernel.org/pub/software/scm/git/docs/git-fast-import....

[3] I actually use the re module [4] as I find the results more readable, and only use the LPeg module when I need something that the re module can't provide.

[4] http://www.inf.puc-rio.br/~roberto/lpeg/re.html

My problem with Lua DSLs is the verbose function syntax (`function foo() return end`). I'm thinking about experimenting more with Metalua to see if it helps in that regard.

One thing that positively impressed me were coroutines though. You can do some very interesting DSL things with it.

Don't you think that with lisp you also need to bend your problem to the particular of lisp? I speak fluently three human languages, and I also have to bend to them when I want to express myself. The more proficient I am the less the bending is noticed, but it is still there.

I'd say it is the same with all good computer languages. You can express whatever you want in python and lisp, but following different grammar.

Totally agree with division into modules lowering complexity (provided it narrows inter-module communication). An interesting idea, that a DSL is way to do this.

But designing a language is hard, including a DSL. Even for experts. e.g. Codd had two attempts at a language for relational algebra, but "mere mortals" couldn't use them. Mathematicians create new notations at the drop of a hat, giving their own definitions to common terms willy-nilly, but this can be an impediment to collaboration.

OTOH, there's an advantage of having many different attempts at a language, inspiring and cross-fertilising each other, til eventually settling on a near-optimal design (if everyone designs their own DSL, this doesn't happen). In practice, you get one group of people experimenting like crazy - academics, visionaries, hackers - and a huge majority that waits to see if something emerges that they can standardise on. Meanwhile, experiments continue. (This is basically the "technology adoption lifecycle".)

The ruby community develops new DSLs very quickly, in the form of frameworks. I get the impression the rapidity of change is challenging, but the improvements are worth it.

I am studying engineering, which is all about OOP. HN is making me so uncontrollably curious about the functional stuff!

Try it. Its not the silver bullet it sounds like when you listen to people rave about it, but it is worth learning.

Keep in mind that functional isn't better or worse than OOP. What you see on HN is merely a reaction to the trend in the past decade (a trend still going on today) to teach OOP as the only de facto paradigm to use.

If Functional Programming was mainstream, everyone here would be raving about OOP and its benefits.

Ultimately what you want is to borrow the good ideas of different paradigms to create elegant and practical code.

Keep in mind that functional isn't better or worse than OOP. What you see on HN is merely a reaction to the trend in the past decade (a trend still going on today) to teach OOP as the only de facto paradigm to use.

To put it another way, OOP is a pretty nice hammer, but so many people have been using it to bash screws in (including ridiculous cases like concrete!) that others have reacted by using screwdrivers for everything (including nails). Pick the right tool for the job.

> Keep in mind that functional isn't better or worse than OOP.

Ah yes, the ol "it's all relative" argument.

if you're at all interested, I can't recommend (http://learnyouahaskell.com/) highly enough.

Would you recommend that or pg's On Lisp?

Learn You a Haskell. I found it easier to read and often thats what you need when your mind is being warped by the content.

Put it in your toolbox. You can apply functional concepts in basically any high-level language.

I think you're wrong about Python. It has very powerful abstraction and runtime reflection mechanisms that enable DSL-like frameworks, such as Flask or Django, to be implemented in it.

Evidence to the contrary:

* Heterogeneity of object abstractions: in order of complexity, you use namedtuples, classes, and metaclasses. In Lisp you'd use the same kind of abstraction across complexity levels.

* Similarly, there are lambdas, plain functions, and bound methods, all for pretty much the same thing.

* Which do you use when? How hard is it to change? http://lukeplant.me.uk/blog/posts/djangos-cbvs-were-a-mistak...

* The way so many configuration variables in Django are strings, that just "coincidentally" point to a class, module, or function.

I've been using Python for 10 years, and Django since pre-1.0. My current startup is built on it. The parts start to show.

Actually, what I like about Python frameworks is that they don't look like DSLs, they manage to be clean, explicit, casual and understandable Python code, even if you may consider them a DSL at a conceptual level.

LOL.

You can bend your statement to your language, or your language to your statement. All essays look the same, regardless of the statement being made. English, Chinese, Persian.