Hacker News Reader

Comment by PaulHoule | Hacker News Reader

It’s little appreciated how much current parser generators are holding the industry back. That is, LISP maintains a lead in metaprogramming because it bypasses Chomskyism alltogether. It really should be a few lines of code to add an “unless(X) {}” statement to “if(!X) ()” to Java, Python, Ruby but the very idea that you could patch an existing grammar with a separate file is like technology that fell off a UFO. Also anything that you generate a parser for should automatically generate an unparser. Sphinx is a fraction of the framework it could be because it can’t process markup and turn it back to Sphinx.

I think a PEG framework with a few features (like an easy way to implement operator precedence just by stating it with either numeric values or X > Y statements) could be revolutionary. Python is almost there but not quite.

View on HN

I used to strongly agree with this. This is an article I wrote on the subject 10 years ago: https://blog.reverberate.org/2013/09/ll-and-lr-in-context-wh...

I even had ideas like what you mentioned about encoding precedence and associativity explicitly: https://github.com/haberman/gazelle/blob/a12a123129dfb7e1f3e...

But I'm no longer as optimistic. The main problem is that nearly all languages have syntax that cannot be easily formalized using declarative abstractions like CFG or PEG, and must fall back to imperative code. And once you have a mix of imperative and declarative code, most of the benefits of a purely-declarative abstraction go away -- or at least the benefits that mattered to me.

If you’re into language parsing and how its patterns could be used in a modern programming language you should take look on Raku (fka Perl 6): It has native support for „Grammars“ [1] which are basically specialized classes to put Regexes in for defining tokens and also combinations of such tokens. Once you‘re done you can use a grammar object to parse text returning an AST object. Raku‘s own syntax is defined in Grammars being a subset of Raku‘s syntax.

So in a nutshell Raku‘s „Grammar“ construct is like RegExes on steroids and renders defining DSLs or other special purpose languages so much easier than in any other modern language.

[1] https://docs.raku.org/language/grammars

I think in most cases metaprogramming makes applications much harder to read. When taking over someone else's codebase, I don't want to also have to understand how they modified the grammar of the language.

Obviously, this isn't a blanket statement - metaprogramming has some amazing success stories (homebrew and vagrant both have excellent DSLs), but more often than not I appreciate code that's just "one layer" of abstraction.

Classes and methods/functions alter the grammar just as much.

Custom grammars are a cool idea. They are terrible in practice and one of the biggest reasons why Lisp failed to take off, why it's almost noone's first choice of language for building something big.

The reason is that new grammar creates a new language, and the divergence of the new language from the base language creates a cultural barrier that inhibits communication. It's harder for new engineers to be productive, and it's harder to collaborate.

Customizing your grammar works best if you're a lone wolf, a one man band. You can lever up your productivity by custom-designing something perfectly suited to both you and your preferred problem domain. But nobody else will understand it.

>> It really should be a few lines of code to add an “unless(X) {}” statement to “if(!X) ()” to...

Or even one line :)

Factor...

  : unless ( ? quot -- res ) swap [ drop ] [ call ] if ; inline

Rebol/Red...

  unless: func [expr block] [either expr [] block]

The difficulty isn't `unless(x) {}`, it's `{} unless(x)` and `{} if(x)`.

Except you also don't have the brackets to keep things organized.

Do you mean semicolons and whitespace?

No, I mean that in the examples of difficult to parse lines there are no {} anywhere and no () on the condition.

  def save = File.write(name, self.to_yaml) unless invalid?

and you could also have

  def save = File.write(name, self.to_yaml) if valid?

If you had to bracket the expression that postfix if/unless applies to, it would be significantly easier to parse.

Reply on news.ycombinator.com