Feature Macros, an Alternative to Feature Expressions in Clojure | Hacker News Reader

Feature Macros, an Alternative to Feature Expressions in Clojure

github.com

80 pointswooby32 comments

I was just talking about the potential problems of FX last night - very interesting to see such an impressive and simple alternative approach. I'm not sure what the drawbacks are, but looking forward to seeing the discussion that grows from this.

I'm curious what problems came up in your conversation?

Many of the most useful applications of feature expressions simply will not work as macros. You'd need to create one new feature-cond enabled macro for each other macro use case. Feature expressions, on the other hand, are a general solution that can be used at the call site without having to reinvent every macro that may ever need platform-conditional behavior, like your ns+ macro.

Consider the proposed +clj form in the body of an extend-protocol or a deftype:

    (defmacro +clj
      "Form is evaluated only in the JVM."
      [form]
      (when (= :clj *host*) form))


    (deftype Foo

      SomeCommonProtocol
      (aMethod [blah blah] ...)

      (+clj
        JvmOnlyInterface
        (someMethod [blah] ...)
        )
    
      )

Of course, this does not work. The deftype macro gets the +clj form unmodified and must handle it on its own.

At best, you could have a syntax-quote style wrapper form:

    (feature-quote
      (deftype Foo

        SomeCommonProtocol
        (aMethod [blah blah] ...)

        (+clj
          JvmOnlyInterface
          (someMethod [blah] ...)
          )

        ))

However, this is essentially what Feature Expressions are anyway. Only broken, since macros don't only operate on syntax/edn, they can operate on any compile-phase value, like dates and times or the output of any tagged literal handler. Without being part of the phase before the macro expander, you can't prevent platform specific types from reaching the macro body.

The reality is that Clojure, in the tradition of most lisps, has rigid read/compile/run phase separation where each phase has a varied evaluation strategy, but the phases run interleaved. Unlike say something like Mathematica, which does not run the reader interleaved with the evaluator - and the evaluator uses normal-order rather than applicative-order! Without such a uniform evaluation strategy (nevermind Mathematica's evaluator's other problems) it simply doesn't make sense to fight so hard against beefing up the read phase.

Related: I've written before about how I don't think that this impacts tooling as badly as you guys thing it does. I just don't understand why you guys are so against feature expressions, nor do I think you've succeeded in proposing a desirable alternative.

What's wrong with:

    (deftype Foo
      SomeCommonProtocol
      (aMethod [blah blah] ...))

    (+clj
     (extend-type Foo
       JvmOnlyInterface
       (someMethod [blah] ...)))

Can you come up with another example you consider intractable?

Your change is invalid. I picked my example very carefully: You can not extend an interface to an object after the type has been created. That only works with protocols.

That said, there are countless examples. I already mentioned the proposed ns+ form, but there are a couple more in the design spec here: http://dev.clojure.org/display/design/Feature+Expressions

I'm not going to bother constructing more concrete examples, since it's trivial to do and each and everyone will have case-specific solutions that will seem "nicer" than feature expressions. But that's just it: It will be a case-by-case solution, rather than a general solution. Case-by-case application-centric solutions are preferable for the platform abstractions of your major subsystems, but often you just want to hack it and feature-macros simply don't offer the flexibility you need.

Ah, point taken. Here is a valid version:

    (case-host
     :clj
     (deftype Foo []
       JvmOnlyInterface
       (aMethod [blah blah] ...))
     :cljs
     (deftype Foo []))
    
    (extend-type Foo
      SomeCommonProtocol
      (someMethod [blah] ...))

It is true there are countless examples. There are also countless counter-examples. And for the examples that are competitive in terseness, there is already cljx.

I personally don't "just want to hack it", especially when it comes to writing portable code. I agree that portability poses subtle new challenges to application design. I just prefer to meet those challenges in a way that is programmable, which syntactic extension is definitely not.

Let me rephrase "just hack it": Feature expressions are a low-level primitive for platform switching. If you have cross platform abstractions, consumers of your code shouldn't need to use feature expressions. However, they are very useful in the implementation of such abstractions.

As for the programability complaint: I just don't see why it is important. If you are generating code, you can already add/remove content from the generated code. That is: you can always just write a "feature macro"! Feature expressions don't add any power you don't already have: Only affordances for humans.

Unrelated thought: Haskell/GHC uses the C preprocessor to address this problem.

I agree that Feature Expressions don't add power, because yes, they are semantically equivalent to C preprocessing and orthogonal to the idea of Lisp.

What excites us most about Feature Macros is that they do add power. Here is an example of something we think is powerful from the proposal - a cross-platform macro:

    (case-host
      :cljs nil
      :clj  (defmacro my-macro [name & body]
              (case-target
                :clj  `(.println (System/-out) (str (do ~@body)))
                :cljs `(.log js/console (str (do ~@body))))))

1) I don't understand what this example is supposed to demonstrate.

2) By "power" I meant a formal notion of expressiveness: Neither feature expressions nor feature macros add anything to the language that you can't already express with existing constructs. Like I said: It's about affordances, not capability.

The example demonstrates how you can define a macro that runs its code in Clojure but can emit code to both Clojure and ClojureScript. How would you do this with feature expressions?

  #+clj
  (defmacro my-macro []
    (cond
      (contains? *features* :clj)  `(some-clojure-thing)
      (contains? *features* :cljs) `(some-cljs-thing)))

If that was too verbose for you, you could of course write a `feature-case` function that looks almost identical to what you have in that example.

This was a great example, and I think you are absolutely right that it is the identical in function, if not in form, to mine.

Interestingly, in the course of concocting a counter-counter example, I came up with this in an effort to show the two dynamic variables we stipulate - host and target - were absolutely required:

    #+clj
    (defmacro my-macro []
      (cond
        (contains? *features* :clj)  `(some-clojure-thing)
        (contains? *features* :cljs) `~(do (require 'some-ns)
                                           ((resolve 'some-ns/some-fn)))))

My hope was to make the point that 'some-ns/some-fn would run in a context where (features :cljs) is true - as it is in the caller's (macro body) environment - which would result in the system attempting to run ClojureScript code, which would explode because the macro is Clojure. Then I realized that require runs load, and load is in the macro body. The load/require available in the environment is Clojure-specific! Thus, load/require can bind disj :cljs and conj :clj before reading and compiling some-ns.

My counter-example failed, but did bring me a little closer to the truth - that we only need one dynamic variable, platform. Any platform that has the ability to load code also has an opportunity to bind platform and thereby inform macros of target. This simplifies the Feature Macro proposal by half and we are excited to amend it.

Our updated cross-platform example becomes:

    (case-platform
      :cljs nil
      :clj (defmacro my-macro []
          (condp = *platform*
            :clj  `(some-clojure-thing)
             :cljs `(some-cljs-thing))))

Instead of host and target, we are down to just platform. We continue to see no need for a features set because of Clojure's platform-symbiosis.

That's not equivalent to inline protocol implementation in both Clojure or ClojureScript. The point still stands, the proposal runs afoul of macro composition issues. This isn't to pass judgement on the proposal, but this is a tradeoff that is downplayed.

wow much high horse, so snippy, much snark

Given you find the lack of "standard in-memory representation for tagged literals" [1] a problem, I'm wondering why you are so set on adding yet another feature with the same problem and don't understand why some might also be looking for an alternative.

[1] http://dev.clojure.org/display/design/Representing+EDN

I didn't intend to be snippy or snarky, and I don't really think I was. I was simply pointing out the problems with this proposal, which has been proposed numerous times by numerous folks, but such proposals never addresses the tradeoffs. If you're willing to chat off-list about my communication style, I'm happy to make tone corrections in the future.

As for the "Representing EDN" design doc that I wrote. You're right to note similarity between tagged literals and feature expressions, however, I don't believe that they warrant the same tradeoff. One stated design goal of EDN is for intermediate processors to be able to handle self-describing data. They can already do that by defining a default tagged reader function. All I'm proposing there is that there be a single standard "uninterpreted" type, so that multiple intermediate processors can cooperate in handling self-describing data in the absence of special purpose tagged readers. Otherwise, each reader would need to perform an eager walk of each object to replace unknown types, and, by that point, they have no way of knowing what types they need to look for to replace in general.

By contrast, Clojure's reader already loses information necessary to represent Clojure source code as opposed to EDN data. The EDN data model is not rich enough on its own to represent code.

I didn't intend to be snippy or snarky ...

Fair enough; some of your comments here appeared to me to be dismissive and my perception was wrong. I should also say I want Niskin and Dipert's efforts here to be taken seriously and I don't want my comment to distract, and for that I apologize to them.

So are there still things that "simply will not work as macros"? If so it would be interesting to see examples.

I do like the idea of keeping them regular forms as it feels more future proof than cljx style expressions.

It seems to me that Feature Macros will be trivially implementable as a library once Feature Expressions land:

    #+clj (def *host* :clj)
    #+cljs (def *host* :cljs)

They're trivially implementable without feature expressions, as we demonstrated with our 20 lines of code.

Right, but if feature expressions are part of "Clojure the language" then feature macros can be implemented as a library. The reverse wouldn't work. Ergo, as attractive as I find feature macros, I think this argues that feature expressions are more "fundamental" and therefore deserving to be part of the core language.

Except on Windows due to Boots current lack of support.

The proposal is not dependent on boot, only our demonstration of it is.

I am looking at one of the examples:

https://github.com/feature-macros/clojurescript/blob/feature...

But if I run Clojure on the JVM, the following fails:

     (def host nil)
     (defn foo [] (when (= :cljs host)
                    (gstring/format "something")))

... the error being that there is no "gstring" namespace.

Since the proposed solution with macros would produce such code (I am right?), how does it solve the issue of namespaces that only exist in a specific platform?

I think the `case-host` macro is what you're looking for: https://github.com/feature-macros/clojurescript/blob/feature...

Ok, thanks. So unlike Common Lisp, symbol resolution does not occur at read time.

Does this also affect the #_ discard macro? And what about anonymous functions like `#(+ 5 %)`? Since those are reader macros I assume yes?

The Feature Macros proposal is only adding macros and vars to Clojure---the reader isn't modified at all. Since all of the things you mentioned are reader macros (as you pointed out), Feature Macros never even see them, so they won't be affected in any way. (Reader macros are expanded before regular macros are, so regular macros don't see things like `#(...)`, they see `(fn [] ...)`.)

The great point made in the README is that we can't generate feature expressions. With feature macros we can have macros that generate cross-platform code. That's big in my mind.

That hasn't been a problem with reader macros in the past since you don't need to generate `#(...)` if you have `(fn [] ...)`.

See my comment here: https://news.ycombinator.com/item?id=8924227

I don't understand when I would ever want to generate a feature expression. The goal is to generate platform-specialized code. You never have to generate code that generates platform-specialized code!

a clean and simple approach to the problem, +1

tldr:

    git clone git://github.com/feature-expressions/clojurescript
    cd clojurescript/feature-macros-demo
    make deps
    make demo

Reply on news.ycombinator.com