Writing a DSL with Python | Hacker News Reader

Writing a DSL with Python

blog.fmeyer.org

43 pointsjgalvez21 comments

I would recommend pyparsing for this. There was an excellent article on it in Python Magazine a few months ago.

Some links:

1. O'Reilly mini-book on pyparsing (probably best single ref) http://oreilly.com/catalog/9780596514235/

2. Quick example of pyparsing http://eikke.com/pyparsing-introduction-bnf-to-code/

3. Main pyparsing page (i know it's not very aesthetically pleasing. The code is much better, I promise) http://pyparsing.wikispaces.com/

4. Links to more on pyparsing here http://pyparsing.wikispaces.com/Publications

Python Magazine is excellent, by the way. Definitely worth the purchase price as they include a zip file with code. (Mandatory disclaimer: I have no stake in it at all. Just found it very useful, that's all).

Note that there is a world of difference between an in-language DSL and one with its own syntax that must be parsed, so much difference that they really should have different names. The former can effortlessly exchange data with and enjoy all the features of the host language. I'm convinced that the latter will one day be an artifact of less enlightened times.

By convention the in-language form is called an internal DSL, the other an external ditto.

Or embedded.

or Pidgin ;-)

ref: http://www.bofh.org.uk/2007/08/08/domain-specific-pidgin

A very interesting DSL for network packet manipulation is Scapy(http://www.secdev.org/projects/scapy/).

It uses python itself as a DSL to make things like packet forging easy:

  >>> Ether()/IP()/TCP()/"GET / HTTP/1.0\r\n\r\n"
  <Ether type=0x800 |IP frag=0 proto=TCP |<TCP |<Raw load='GET / HTTP/1.0\r\n\r\n' |>>>>

Here the / operator was overloaded to stack layers.

Wow. If you ever wanted curly braces or multi-line lambdas in Python this looks like the way to do it.

What exactly would people do with multi-line lambdas? I would definitely refactor any lambda with statements or more than one expression in it, give it a name, or use a list comprehension, so I've never seen a problem with them. Can anybody give me an example of a piece of code they think would look better if python had multi-line lambdas?

If you give something a name, yet still only use it once, what's the point? It is just semantic clutter to do that (unless, of course, you can't use multiline lambdas...).

There is no big harm in giving something a name. Giving a name also helps if you find a need to reuse it later. Plus a name can give a hint to what this code is for.

Yet, if you look in functional programming code (e.g. Haskell, Scheme), you'll find many anonymous lambdas (that in python would be > 1 line). Why would good coders bother doing that?

If something is small and enough to be anonymous, it's usually simply enough to be expressed without binding a variable.

(See point-free style.)

Obviously. Show us how to do point-free style in Python. We're talking about Python's lame implementation of anonymous functions, not something that it doesn't have any implementation of.

Oh, I thought you were talking about programmers' habits in Haskell.

In Python, you can easily define methods within methods, so there is little semantic clutter at the toplevel.

  def doSomething():
    def doSomethingElse():
      # Code goes here.
      pass
    
    eggs = filter(doSomethingElse, spam)
    for cheese in eggs:
      # Do something.
      pass

Those are just two of the complaints I hear most. A friend of mine actually went so far as to patch Python to use curly braces at the start of a project in Python. "Perfect Python" he called it. Personally I think deviating from PEP8 & standard idioms is likely to cause more pain in maintainability than you win in elegant syntax.

Ugh, he means imperfect python, right? Even if I hated the indention sytnax, which I dont (and it seems to me that it grows on most people after a few weeks of use), I think a patched python is the wrong way to go about it as you now deviate from the main python distributions, so cant make use of updates and bugfixes (and do all libraries play nice with a patched version? maybe - maybe not). If you absolutely must have braces, use #{ and #} :-D, you should be indenting anyway!

I really don't get why some people hate the indention-as-syntax. Sure, its different to C and Java, but it also results in (more) consistently formatted code and readable indention. It also only takes a day or two to get used to, even if you're a hardened C++ user (for the record, I have nothing against C/C++ syntax, I actually switched from years of C++ to Python as my main language and I just recently started a C++/Python hybrid project).

Because functional languages discourage loops. This is just a really trivial example, but:

    # value x =[|1;2;3;4|];
    value x : array int = [|1; 2; 3; 4|]
    # for i = 0 to (Array.length x) - 1 do {
      Printf.printf "%d" x.(i);
      };
    1234- : unit = ()
    # Array.iter (fun y -> Printf.printf "%d" y) x;
    1234- : unit = ()

Basically the entire body of any for loop I would want to make an anonymous function. Then if I cared about the result I could do:

    # Array.map (fun y -> Printf.printf "%d" y) x;
    1234- : array unit = [|(); (); (); ()|]

Assuming the underlying platform supported it, and the function is pure, that's "free" parallelism.

from __future__ import braces

antlr, antlr, antlr

Maybe provide a link next time?

http://www.antlr.org/

Reply on news.ycombinator.com