Magritte: A Language for Pipe-Based Programming [pdf] | Hacker News Reader

Magritte: A Language for Pipe-Based Programming [pdf]

files.jneen.net

114 pointsself43 comments

I must make a mandatory fanboi shoutout to: https://luna-lang.org

This looks seriously cool and partly resembles how I think computation should be in that it is a series of steps that meat together in a data pipeline.

Great concept but I found their execution lackluster. The studio (IDE) is an Atom-based editor that is horribly slow and very buggy. I'm still waiting for a cleaner, smoother implementation. Honestly, if they were reinventing the GUI, might as well be an extension for VS Code.

VS Code is not universally extensible. Atom is. Extensions in VS Code have a lot more limitations. This is also in part because VS Code was designed to be more performant than Atom.

The problem with all these data flow languages is they don't have control flow.

there is dataflow with some level of control flow (boolean conditions, boolean loops, and event-driven conditions) in https://ossia.io - it's tailored to artistic uses however :-)

the details are in there : https://tel.archives-ouvertes.fr/tel-01947309/document

That looks interesting! I've been using pure data on and off for a while. It seems to be the most complete one so far.

yes, puredata is excellent and a huge inspiration :-) (its author, Miller Puckette, was in my phd jury !)

Holy moly! Curses in hex out of envy :)

In this linked document for Magritte, it describes its pipeline operator in a section called "Syntax Design - Linear Composition: Why Not Lisp?".

..The problem is the prefix-only calling convention that requires a back-and-forth movement of the cursor, and causes the expression to expand leftwards, opposite of the direction of the user’s typing.

While some languages have some support for linear composition using operators such as Clojure’s arrow macros or OCaml’s |> operator, the integration of these macros with the underlying subsystem is shallow.

I question this last statement.

Just this weekend I was reading about Hy's (based on Clojure's) threading macros:

"Threading macros, also known as arrow macros, convert nested function calls into a linear flow of function calls"

-> (or the threading macro) is used to avoid nesting of expressions. The threading macro inserts each expression into the next expression’s first argument place.

->> (or the threading tail macro) is similar to the threading macro, but instead of inserting each expression into the next expression’s first argument, it appends it as the last argument.

At least to me this seems far from "shallow" and even more useful/flexible than the pipeline operator.

Yeah basically. As a ex Clojure user, I’d agree. It’s not shallow and if one finds it shallow then one can write their own macro to be less shallow.

As I'm educating myself on Lispy languages, may I ask why the "ex" for Clojure and what (if any) other language you've moved onto?

I've always used many languages, but I tried clojure for data processing stuff. I moved away from it because the number of libraries was tiny; I had to write my own libraries for doing basic SQLy things. Some hard things were easy in Clojure, but too many easy things were hard.

Thanks! That was interesting to hear about your experience with Clojure. I've heard good things about the language, reading here and there about its features and strengths - so I was curious to know if there were any critiques/cons (pun intended :).

This looks cool! I linked it on my wiki page of alternative shells:

https://github.com/oilshell/oil/wiki/ExternalResources

(which is cited on the first page of the thesis!)

I hope that Oil can a lot of these things in a backward compatible way, but I still have to read it and understand it more.

You may consider adding to awk-like group of programs pawk:

https://github.com/alecthomas/pawk

It gives you a more readable and concise way to select a colum, split it and sau snip out few characters.

The twist here is that this language doesn't use actual pipes; it just creates representations of pipes that can be used by a different interpreter.

So you're saying this is not a pipe?

Appreciated :)

> The implementation is written in Ruby, and the source code is publicly available at https://github.com/prg‑titech/magritte.

Somehow the link had CGI escaping in it and I ended up on the 404, hopefully this one should work https://github.com/prg-titech/magritte

Thanks for sharing.

It was copied directly from the PDF and contains a Unicode hyphen U+2011 encoded in UTF-8. I should have checked for such and replaced it.

    1.4 The Desktop-Scripting Problem
    
    Related to the shell-language design problem is the desktop-scripting problem:
    How should unrelated programs written in different languages be integrated—especially
    in an ad-hoc manner in a desktop environment? Such a task can require a large
    amount of glue code, written by users who are unfamiliar with the inner workings of
    the programs they are using. Notable attempts at solving the desktop-scripting prob-
    lem include the TCL language [Ousterhout, 1989] and Guile Scheme [Blandy, 1998].
    However, while most of these approaches use a large, robust language, it still remains
    difficult to integrate them with external programs—instead putting the burden on
    those programs to integrate with their system. We believe a shell-based approach is
    promising

Hmm, Scheme as a large language? Well, I don't know much about the extensions Guile Scheme has over RxRS, would hope for someone to add info here.

Separate from any implementation specific extension, the various RxRS can be considered small or large, too.

This is only a partial answer, but I remember the designers of Julia saying that the Scheme spec spends many dozens of pages on the semantics of integers and floats. That makes the language large.

Remember that Scheme is not the lambda calculus -- you need dynamic types like integers, strings, vectors, hash tables, IO, etc. to get anything done.

In contrast in Julia they said that the semantics of numbers are defined in Julia. Julia has a much richer notion of number than any other language AFAICT, e.g. int8 .. int128, uint8 .. uint128, vectors of those, matrices of those, rationals, etc. They're all on the "same level" -- there aren't primitives vs. libraries, as far as I know.

https://docs.julialang.org/en/v1/manual/integers-and-floatin...

https://docs.julialang.org/en/v1/manual/complex-and-rational...

https://docs.julialang.org/en/v1/manual/arrays/

https://docs.julialang.org/en/v1/manual/conversion-and-promo...

It is mind boggling that despite the name, they did not (given the (I assume) joke with the name) say, "Ceci n'est pas une pipe" anywhere in the paper. Only once does the phrase, "This is not" occur, but completely out of context of Magritte's most famous painting.

I'm serious - this isn't Joke 101, but this is Marketing 101 in some respects.

For context, this is the painting referenced above: https://medium.com/lefkosh/art-analysis-this-is-not-a-pipe-d...

Perhaps that's best left as an exercise for the reader.

I assume they felt it was a bit too on the nose? Anyone familiar with Magritte will be familiar with the painting; any attempts to shoehorn the quote in there would have felt forced, I think. Maybe if they had a website to go with it... but this is just a paper.

Plus, as jpfed says below: The twist here is that this language doesn't use actual pipes; it just creates representations of pipes that can be used by a different interpreter.

I wrote clojure for a while. Forward chaining basically facilitates pipe based programming. When it fits it’s great, but when it doesn’t try code is ugly.

Some computations just want to be imperative. IMO, if you can avoid developing programming fetishes; power to you.

Looks interesting!

Side note: here's another "Magritte" from 1983: http://reports-archive.adm.cs.cmu.edu/anon/scan/CMU-CS-83-13.... Is there a connection?

Yes, both look very interesting!

Here's another one: the Magritte meta-description framework for Squeak Smalltalk, often used with the Seaside continuation-based web framework:

http://www.hpi.uni-potsdam.de/hirschfeld/seaside/tutorial?ch...

Here is one for R:

https://github.com/tidyverse/magrittr

Yep, pipes in R are great for interactive use & EDA. (Not necessarily for production code.)

Reminds me of nushell, which pipes around 2D tables (which can instead store hierarchical structured data, for example when reading toml files): https://news.ycombinator.com/item?id=20783006

Except for the comparisons with Lisp in the third section, I thought the idea of pipes with complex data structures to be a pretty neat idea. Also, this is for a Master’S Thesis from the University of Tokyo? That looks like a lot of work for a MS thesis.

Taking "pipe-based programming" to its logical conclusion results in http://www.linusakesson.net/programming/pipelogic/

The problem as it's defined in this thesis sounds a lot like the problem of microservice architecture. I'm curious how something like protobufs would compare in solving this problem

Mario provides an implementation of shell pipes for Python.

https://github.com/python-mario/mario

Please stop spamming that project all the time.

Just remember to pipe with the | character. ¦ n'est pas une pipe.

But that would ruin the joke - surely in Magritte

"| ceci n'est pas une pipe"

This is not a programming language

Reply on news.ycombinator.com