Hacker News Reader

> 4) Almost every "old" computer language has slowly moved away from unprincipled metaprogramming precisely because it's difficult to reason about, for both humans and computers. (C, JavaScript, and Python, to name a few, all have unprincipled metaprogramming facilities, which have been, over time, relegated to specific use patterns to maintain coder sanity.) What do we hope to gain from a new language whose raison d'être is based on this feature?

This complaint makes little sense, because macros are expanded entirely at compile-time. Reasoning about a macro is no harder than reasoning about a higher-order function: you reason about the macro function (which is a simple transformation on some tree data structure), and the generated function. Because a macro is always expanded at compile-time, you can always expand it and inspect the generated function.

I wouldn't use Javascript and Python as examples here--those languages, suffering from their lack of meta-programming, instead encourage heinous things like monkey-patching and runtime introspection, which are insanely difficult to reason about.

Compare: what I might do in Common Lisp by writing a set of macros (and checking the expansions--just a hotkey away in my IDE) I'd do in Python by adding fields and methods to objects at runtime.

Reasoning about a macro is no harder than reasoning about a higher-order function: you reason about the macro function (which is a simple transformation on some tree data structure), and the generated function.

Except that C-style macro functions aren't "simple transformations on some tree data structure". They are textual transformations. Hygienic macros (à la Racket) I agree are useful and easy to reason about.

I wouldn't use Javascript and Python as examples here--those languages, suffering from their lack of meta-programming, instead encourage heinous things like monkey-patching and runtime introspection, which are insanely difficult to reason about.

I agree that the mechanism underlying their meta-programming facilities is treacherous. What I mean (and didn't state clearly) is that over the decades, these languages have developed patterns for reining in the power of their facilities (decorators in Python, and a fairly standard pattern for emulating class-bassed OO in Javascript).

> Except that C-style macro functions aren't "simple transformations on some tree data structure". They are textual transformations. Hygienic macros (à la Racket) I agree are useful and easy to reason about.

Elixir uses the latter; am I missing something?

1) The language introduction is found under the Getting Started link: http://elixir-lang.org/getting_started/1.html

2) Dialyzer works fine on Elixir beam files. You can specify type & spec attributes just like in Erlang.

If macros worry you unduly, then Elixir is not for you. I personally prefer to use macros than type lots of boilerplate code, but yes there are some downsides.

edit: Actually I should have said reading and typing - reading boilerplate is much worse than writing it. Judicious use of macros can clarify intent for someone reading the code.

1) A language introduction is not documentation, and is certainly not specification. It's difficult to discuss a new language when you have to guess how it works based on a handful of examples.

2) Dialyzer is a complete, not sound program analyzer. i.e. it will never produce a false positive, but it may allow incorrect programs through. By making definitions first-class, Elixir's module language becomes Turing-complete, moving a whole new class of bugs into Dialyzer's "false negative" category.

This problem doesn't exist with second-class definition systems such as OCaml's, because the definition language is not Turing-complete: all definitions and modules may be checked at compile time.

3) I'm not sure where macros come into play? Are you using "macros" to mean "higher-order programming tools"? If so, you're misunderstanding me -- I use metaprogramming tools all the time; I simply prefer ones which aren't Turing-complete and are therefore amenable to analysis.

> It's difficult to discuss a new language when you have to guess how it works based on a handful of examples.

It sounds like you are more interested in "CS friendly" languages, rather than "get stuff done" languages. Erlang and Elixir tend toward the latter category.

I'm generalizing a lot, and that's not to say a language can't be both, but Erlang was born of a practical need, and I think that Elixir is similar: it's not meant to be the most beautiful language out there, it's meant to be a bit more user-friendly way of utilizing the power of Erlang and OTP.

In terms of a specification, have a look at Elixir's test suite. Given that it's still being hacked on, it's probably not set in stone.

It sounds like you are more interested in "CS friendly" languages, rather than "get stuff done" languages.

It sounds like you think there's no overlap (despite your next paragraph).

Additionally: "CS friendliness" is more pragmatic than you think. Automatic refactoring and automatic code analysis are two huge pragmatic wins (think: saving developer time) that are made possible only by paying heed to analyzability of code.

Erlang and Elixir tend toward the latter category.

Erlang is my current favorite language. You may not realize it, but it is one of the most "CS friendly" mainstream languages that exist. The fact that it is almost entirely defined by "functional programming" + "actor model" is a big part of this. The existence of Dialyzer is a testament to its amenability to automatic analysis.

In terms of a specification, have a look at Elixir's test suite.

A test suite isn't a specification (though it's a lot closer than a tutorial): all computer languages contain infinities (e.g. expressions can be arbitrarily large); test suites cannot express infinities.

And a test suite certainly isn't a reference document. A good language reference document says "here are all the forms of the language, here is what they all mean". The Erlang Reference Manual is a very good example of such a document.

(I'm not trying to be a bear; "There is no language documentation" is a perfectly acceptable answer for me; it just means there is no way for me to discuss the specific benefits Elixir may have.)

> The Erlang Reference Manual is a very good example of such a document.

Erlang has had quite a bit longer to produce such a document, though, hasn't it?

Why bother doing that when you're still fiddling with the language?

In terms of CS friendliness, I get the impression that Haskell is far more popular with that crowd in that it has more things of interest to people doing language hacking. Erlang has "functional" and "actors", but not a lot of other stuff that seems to be of interest to the CS crowd, like types.

> "There is no language documentation" is a perfectly acceptable answer for me; it just means there is no way for me to discuss the specific benefits Elixir may have

No way at all? I think you can tell a lot about a language without some kind of formal documentation.

Why bother doing that when you're still fiddling with the language?

To me, that sounds entirely backward :) I've never implemented a language I haven't already designed "on paper". (Why bother? With a good reference document, you can write and test programs in your head. Implementation is tedious; I only want to do that once.)

In terms of CS friendliness, I get the impression that Haskell is far more popular […]

It is. I don't really know why. Haskell has a lot of libraries and a good compiler, but it's otherwise pretty boring. Its type system is nothing special (OCaml and Mercury have very similar but more interesting systems, notably w/r/t subtyping), monadic I/O is one of the silliest contortions I've seen (uniqueness types in Mercury and Clean are much easier to work with), and laziness isn't all that useful (it is useful but I've never missed not having it).

(I say this with all respect to the Haskell designers: it's a very good language, and they successfully met their self-imposed goal of being fully functional; I just happen to think that that goal led to Haskell being ultimately boring.)

Erlang has "functional" and "actors", but not a lot of other stuff that seems to be of interest to the CS crowd, like types.

That's exactly it. It doesn't have a lot of stuff. The language is incredibly simple, EVERYTHING is explicit, it doesn't encourage first-class metaprogramming, and it ships with its own parser and pretty-printer. That's a CS researcher's dream. That's why Dialyzer (which is better than many, if not most, built-in type systems) can exist. You don't need a built-in type system if your language is simple and analyzable enough to admit a 3rd-party one.

No way at all? I think you can tell a lot about a language without some kind of formal documentation.

Sadly, all I can tell from the tutorial and front page is "Elixir has some kind of metaprogramming that's probably based on Ruby or Javascript". Like I said, I'm a huge fan of metaprogramming done right, but that hinges on lots of specific language details that aren't clearly documented.

Without specifics, any discussion about Elixir's metaprogramming features might as well be a discussion about Javascript's or Ruby's, because that's all I have to go on.

> To me, that sounds entirely backward :) I've never implemented a language I haven't already designed "on paper". (Why bother? With a good reference document, you can write and test programs in your head. Implementation is tedious; I only want to do that once.)

The answer in this case seems likely to be: Jose started fiddling around with it, and at the same time was learning what he could and could not do on Erlang's VM.

I guess I take a favorable view of experimentation and a dimmer view of being able to plan everything ahead of time. At some point in time, things need to be stabilized, but at the beginning, some contact and pushback from the real world and real usage may well impact the design of a system.

> monadic I/O is one of the silliest contortions I've seen (uniqueness types in Mercury and Clean are much easier to work with)

As far as I can see they're all basically the same thing. You enforce sequence by creating a chain of notional "real world" values. I can't say I've used Clean or Mercury extensively, but as far as I can see they just make you pass around that value explicitly. That doesn't really seem like it would be "easier to work with".

> That doesn't really seem like it would be "easier to work with".

1. You dont need monadic versions of all non monadic functions. You dont need fmap, you can just use map.

2. It means there is a well defined order of execution (i.e. The order does not "emerge" as in Haskell, instead it is defined and verified by type system)

It is much much easier to work. In Haskell you feel forcedto separate io from transformation as much as posible. Often you end with either repeating ourself a lot (writing oth mondic as non monadic versions) ir you venture out in monad transformers, which imho is the "goto" of Haskell. (How to turn your code in unreadable unmaintainable spagetti). The cognitive overload is never worth it.

Ps. Uniqueness types are not just about IO. Rust uses them as well. They are about sharing. Haskells needs an IO () type because without a monad it has no defined order of execution, and it cant prevent you from "forking the world" when working with outside references.

If Haskell could prevent you (by using uniqueness) i doubt they would have used monads for IO. (Although they can be usefull in other areas).

> If macros worry you unduly, then Elixir is not for you.

Really? It seems interesting, but I'd simply choose not to use macros in my own code. Seems like they can be avoided in most situations. There's even a heading in the introduction that says 'Don't write macros': http://elixir-lang.org/getting_started/5.html

Macros are more for writing libraries than applications, so certainly what you are saying is correct. However, some people don't like that capability to exist at all - they are afraid it will be abused in code they support or rely on.

sorry to ask, but what do you mean by "unprincipled metaprogramming"? what would be "principled metaprogramming" to contrast it with? (it's really the first time I heard the terms in a macro discussion...)

Ya, I sort of made those terms up, sorry. What I mean roughly is:

* unprincipled metaprogramming: any metaprogramming facility whereby module-level elements (e.g. delcarations, code, modules themselves) may partake in Turing-complete transformations (hence precluding complete analysis), may generate unsound code (i.e. what would be a compile-time error), or may modify code at run-time (i.e. through dynamic scope or global mutation).

* principled metaprogramming: not the above; i.e., all code transformations terminate (allowing automatic analysis), all generated code is syntactically sound (mostly this means not having to worry about escaping stuff), and code transformations don't modify existing code.

There are probably other things and some exceptions, but that's the jist of it.

eval(), C++ templates, C macros, and prototypes are all examples of unprincipled metaprogramming.

Parametric modules (OCaml), hygienic macros (Racket), and multistage programming (MetaOCaml almost) are all examples of principled metaprogramming. (I say "almost" for MetaOCaml because code generation expressions may not terminate; but this is neither required nor encouraged, and it is easy to guard against.)

You are aware that Racket (hygienic) macros are Turing-complete? You can loop and branch, and macro expansion may not terminate too.

I think only OCaml functors really qualify to what you call "principled", but then - it's not the most powerful kind of metaprogramming I've seen.

Mm, last time I used Racket (then PLT Scheme) macros I did not encounter those features… the only "looping" I saw was limited to the "..." construct (which is a finite expansion), and all branching was lexical (i.e. branching couldn't be used to form infinite loops). If I am incorrect then I retract my Racket example.

You're most likely incorrect - I cannot be sure about Racket back when it wasn't called that, but Scheme hygienic macro systems - those created with SYNTAX-RULES - are Turing complete. Look here for a "primer" which explains how to 'loop' (by means of recursion) and branch with pattern matching. At the end of a tutorial you can find basic Scheme interpreter written as a set of Scheme macros.

For example, in terms of this macro system, you can (and you could for a long time) define a simplest non-terminating macro like this:

    (define-syntax loop
        (syntax-rules ()
            ((loop) (loop))))

Racket Macro Stepper gives a following warning when used to step through the macro expansion:

    Macro expansion has taken a suspiciously large number of steps.

    Click Stop to stop macro expansion and see the steps taken so far, or
    click Continue to let it run a bit longer.

Elixir's macros are hygenic. They are expanded at compile time. Other code is not modified. Elixir does have eval, just as Erlang does.

My concern isn't Elixir's macros (which I'm glad to see are hygienic); it's the concept that "declarations are expressions".

Prototypes... that blurs the subject a bit don't you think?

> Where is the language documentation?

The language is documented extensively under the Getting Started section of the website. It's a pretty easy read and really helps to understand the pieces of the language that make Elixir so powerful.

for 4) - ruby style DSLs. It's something that Erlang doesn't lack in principle due to parse transformations, but in practice, the difficulty of writing parse transformations and the sense in which prolog syntax doesn't lend itself to simple "english" like DSLs means people don't use them much.

Maybe it's just me, but I feel that in-language DSLs are one of the best ways to write unmaintanable, bug-ridden code.

Why? Once it's no longer syntactically clear what's written in the host language and what's written in the DSL, it requires a very thorough understanding of both languages and their interactions to comprehend any given piece of code. And in a language where definitions can be modified at run time or have dynamic scope (it's not clear which of these two methods Elixir uses), it's often hard to tell what is or is not DSL code in the first place.

(Note: I write DSLs for a living. Except, they live apart from any host language, and thus don't allow arbitrary syntactic and semantic mixing and are thus highly amenable to automatic analysis, which is one of, if not the, primary benefit of a DSL.)

Yes, this is precisely what I mean when I said "if macros worry you ..." in response to your question about unprincipled meta-programming. If you do not like Ruby-style DSLs then you would not like Elixir's use of macros to achieve similar goals.