Hacker News Reader

Comment by netbioserror | Hacker News Reader

Homoiconicity wins in my book. Clojure is the best proof I have of this: What mattered was not the linked list or the cons cell. What mattered was that all code was just nested calls to functions and macros with the SAME general form: (function arg1 arg2 arg3 ...)

Expression-based programming using this core form is trivially easy, and makes immutable data structures downright pleasurable to work with. It also makes code generation easy, though I've never taken full advantage of it; even so, a DSL made of runtime functions is not only possible, but a natural extension of anything you're already doing. Backing those homoiconic expression forms with typical high-performance data structures like arrays and hash maps, as Clojure and Janet do, is a winning formula for making complex tasks simple.

All the patterns of yesteryear for modeling problems begin to look quaint when your program eventually just becomes a pyramid of expression calls, with `main` at the top and database hits at the bottom. The organization of the stuff in between really just becomes a question of standardizing function signatures and organizing modules so humans can easily navigate them.

View on HN

Embedded in this response is a consideration for how programming languages handle data. In Java, if you have a config file, does that file use Java syntax? No. What is that Java file written in? Likely XML. Can you write XML literals in Java? No. Wouldn't it be nice to:

    XMLElement el = <html><body>Hello World!</body></html>;

In JavaScript, we have probably the closest to Lisp in this regard with JSON. You can define JSON literals in your JavaScript code; config files like package.json are written using this object notation. If you want to work with, say XML, there are libraries to convert your XML to JSON and you can take it from there.

But, JSON is limited. How do you define dates for your datatypes? Probably use a string. In Lisp, you can use a string or have some (date year month day) form. How do you represent rational numbers? Maybe you'll risk using floating point or you'll just go with a string. Lisps have rationals built in or you could roll your own with (rational numerator denominator). So your JSON

    {
      'name': 'John Doe',
      'birth-date': '1970-01-01',
      'account-balance': '1000.01'
    }

can become

    (user #:name "John Doe" #:birth-date (date 1970 1 1) #:account-balance 100001/100)

When Lispers need to interact with JSON, they convert it to S-expressions. When Lispers need to interact with XML, they convert it to S-expressions. Wouldn't it be nice if the other languages had syntaxes that made you want to define data using that language's syntax and not JSON or XML?

Visual Basic .NET actually does have XML literals [1] - one of the few reasons to use it!

[1]: https://learn.microsoft.com/en-us/dotnet/visual-basic/progra...

Scala also had it, but seems they replaced it in Scala 3 with a more general concept[1] as everyone thought it was a bad idea to special case XML (why XML, not JSON or something else?).

[1] https://docs.scala-lang.org/scala3/reference/dropped-feature...

This makes sense but to be honest, how I'd really like to represent data to go along with code is as a table or a tree (i.e. presentation/interaction more like a spreadsheet).

A Lisp response to this might be: tables and trees are just special forms of lists.

You can work with tables, trees, maps, vectors, etc. and if they can support list operations (filter, map, reduce, first, last, etc.) then they can be treated like a list and you can do those operations. You can still build IDEs that will show you tables of variables at runtime and graphs of your tree structures.

It's worth looking at Rebol [1], a language that is also homoiconic which was influenced by Lisp, Forth, Self & Logo and had a big influence on JSON too!

  user: [name: "John Doe" birth-date: 1970-01-01 account-balance: $1000.01]

[1] https://en.wikipedia.org/wiki/Rebol

> had a big influence on JSON too!

It might have influenced Javascript, but the entire idea in JSON is to have a data exchange syntax that can be plonked into Javascript as a literal. That requirement leaves room for no other influence, pretty much.

Here's a video where Crockford mentions Rebol (and others) being an influence on JSON - https://www.youtube.com/watch?v=-C-JoyNuQJs&t=1233s

I've seen it mentioned on Rebol chatter that Crockford approached Carl Sassenrath (creator of Rebol) to open-source & use Rebol prior to creating JSON. So having a Javascript literal wasn't on Crockford's mind at that point.

NB. Rebol was closed sourced until 2012.

Very much agree. It also gets you some really useful structural editing capabilities as a side-effect. I can shiffle code around so easily and blindly by just telling it to move an expression right or left, or adjust parens in or out. I also don't have to learn new weird syntax every time the language wants to do something kinda new. So many new Clojure "language" features are actually just libraries which is awesome. Tooling just keeps working and others can provide competing implementations. For instance clojure.spec

This is the best part. Especially with JavaScript with so much difference in runtime and the importance of babel. Even the `loop` DSL can be broken down to basic lisp structure. The flexibility of the language, by allowing itself to be altered, is the most important aspect for me. After completing a first version of a solution, I often find my mind wandering searching for a cleaner, more understandable version of the code. And this is usually done by molding the language to the problem domain.

Something like React in JavaScript exemplifies this. JSX was added for an easier developer experience, but the issue itself is created by the separation of code and data in the language. And CLOS would be perfect for the Component model.

Although if you want to be maximally pedantic about it, homoiconicity you actually get for free in raw machine code. Though Lisp certainly "rediscovered" it in an abstract language(though Church's lambda calculus is also homoiconic, and directly inspired much of Lisp so I suppose he deserved some credit too).

I sometimes ponder about whether we'd be able to recognise any programming languages from alien civilisations. And I usually come to the conclusion that it would have to be either a lisp or a forth, because at their core they're so dead simple it's hard to imagine not inventing them by accident at some point. The fact that Lisp was invented so incredibly early on in computing history, and at least partially by accident, supports this hypothesis.

> It also makes code generation easy, though I've never taken full advantage of it;

I bet you have because Hiccup is an HTML DSL! Compojure is likewise a DSL for building Ring handlers based on HTTP request types.

But to your larger point, I also find I don't reach for macros at all. Looking carefully, I think this is because we have first-class syntax for vectors and maps, and those containers are very flexible about the type of their contents. Combined with the Seq abstraction and a well equipped standard library most Clojure code I write is macro-free.

Perhaps I'm being pedantic, but `if` and many other control-flow constructs are implemented as macros in Clojure. So you probably use macros a lot!

So everyone uses C.

SBCL is self-hosting. It feels like a bit of a stretch to say syscalls are macros, but if you really want to torture language that way then sure because the kernel is mostly C.

In CL, IF is a special form and COND is a macro.

> most Clojure code I write is macro-free.

Conceptually, most forms that are not on the form (fn args...) are macros. You got like cond, lambda and label forms, that are not macros. I don't think Clojure technically has label forms though.

You are correct. I should have clarified. I meant I don't write many custom macros myself.

In case of Clojure I would argue it is the combination of very consistent - simple some would say ;-) - syntax and stability in the language that partially stems from the simplicity and, as you mention, at the same time extensibility of Lisp's approach "function first, arguments later" and homoiconicity. However, dynamic typing, immutable data structures by default, software transactional memory built into the language itself, stellar interop with Java (or JavaScript in case of ClojureScript) and the REPL workflow that eases things like hot-code reloading are all major contributors and reasons, why Clojure(Script) users are usually quite enthusiastic about the language family.

Don't get me wrong, Clojure does have some warts inherited from the approach of the host platform. E.g. I think its math could do nil pruning by default with e.g. unchecked-add and similar skipping nil pruning and overflow checking (for best performance in some tight scenarios). ClojureScript basically does nil pruning already.

As a side note, I would love if ClojureCLR picked up steam a lot more. I heard .NET is not as dynamic as the JVM which makes these things harder but if ClojureCLR would be on the same level as at least ClojureScript in terms of support and tooling, the Clojure family of languages would be really hard to beat for any kind of business application development. With ClojureDart, Babashka, NBB and to some degree Clojerl, Joker, Jank and others it seems the family extends much beyond the original "business applications" area into scripting, embedded and to some degree HPC or network programming as well. I guess we will see where it catches on.

No better example of this than Racket.

Function application itself is an overridable function. (f 1 2 3) compiles to something like (#%app f 1 2 3). And you can hook into that.

Runtime type checking, stack traces, debuggers..

You used to be able to do fun stuff like that in clojure but it ended up being axed in the name of performance.

(Well, I guess you still can, but the standard library is off limits now basically.)

In most cases, the new definition of #%app is a small wrapper around the standard one, with some special case. If the new definition is small enough, the Racket compiler can inline it and hopefully detect that it's the standard case and you get the same speed at the run time.

In some cases, it may be useful to use a macro to "force" the inlining, but it may increase the total size of the code.

There are a few languages implemented in Racket that do this, https://docs.racket-lang.org/rackjure/index.html https://docs.racket-lang.org/lua-manual@lua-lang/index.html that redefine the #%app and ITRC in most case get the fast application after the compilation.

In any case, if someone has a similar project with a custom #%app that is too slow, hey can ask using github or discourse and I(we)'ll try to help. In some case it's impossible, but in other case some tweaks make the redefinition of #%app more optimizer friendly.

> All the patterns of yesteryear for modeling problems begin to look quaint when your program eventually just becomes a pyramid of expression calls, with `main` at the top and database hits at the bottom. The organization of the stuff in between really just becomes a question of standardizing function signatures and organizing modules so humans can easily navigate them.

You don't need homoiconicity for that; in fact expressions tend to be easier to write in a language that's slightly non-homoiconic. Even Lisp or TCL fans tend to use a macro or similar to embed mathematical expressions, because regular infix mathematics is significantly more readable than Polish notation.

Having a good way to write data literals is important, and the lack of it is a big part of why Algol-family languages are so awful - C/C++/Java/etc. code ends up being stringly typed because it's easy to write literals for strings and a bunch of random number formats, and cumbersome to write literals of anything else. But that doesn't mean your data syntax has to be exactly the same as your code syntax; some similarity is helpful, but the benefits of making your code look exactly like the AST of your code are pretty marginal.

Expressing data and instance declarations in the same syntax you express code and class definitions is an essential feature that enables "Instance First Development", and the "Instance Substitution Principal", as supported by OpenLaszlo, and described by Oliver Steele. And throwing declarative constraint based programming into that mix is really synergistically powerful and expressive.

https://news.ycombinator.com/item?id=21841054

DonHopkins on Dec 20, 2019 | root | parent | next [–]

My remark was just an old Java joke I repurposed for Ant!

"Java is a DSL for taking large XML files and converting them to stack traces." -Andrew Back

https://www.reddit.com/r/programming/comments/eaqgk/java_is_...

But in all seriousness:

OpenLaszlo used XML with embedded JavaScript in a way that let you extend XML by defining your own tags in XML+JavaScript. I've done a lot of work with it, and once you make your peace with XML (which seemed like a prudent thing to do at the time), it's a really productive enjoyable way to program! But that's more thanks to the design of OpenLaszlo itself, rather than XML.

https://en.wikipedia.org/wiki/OpenLaszlo

OpenLaszlo (which was released in 2001) inspired Adobe Flex (which was released in 2004), but Flex missed the point of several of the most important aspects of OpenLaszlo (first and foremost being cross platform and not locking you into Flash, which was the entire point of Flex, but also the declarative constraints and "Instance First Development" and the "Instance Substitution Principal", as defined by Oliver Steele).

https://en.wikipedia.org/wiki/Apache_Flex

https://blog.osteele.com/2004/03/classes-and-prototypes/

The mantle of constraint based programming (but not Instance First Development) has been recently taken up by the "Reactive Programming" craze (which is great, but would be better with a more homoiconic language that supported Instance First Development and the Instance Substitution Principle, which are different but complementary features with a lot of synergy). The term "Reactive Programming" describes a popular old idea: what spreadsheets had been doing for decades.

OpenLaszlo and Garnet (a research user interface system written by Brad Myers at CMU in Common Lisp) were exploring applying automatic constraints to user interface programming. Garnet started in the early 1990's. Before that, Ivan Sutherland's Sketchpad explored constraints in 1963, and inspired the Visual Geometry Project in the mid 1980's and The Geometer's Sketchpad in 1995.

https://en.wikipedia.org/wiki/Reactive_programming

http://www.cs.cmu.edu/afs/cs/project/garnet/www/garnet-home....

https://en.wikipedia.org/wiki/Sketchpad

https://web.archive.org/web/20160303205845/http://math.coe.u...

https://en.wikipedia.org/wiki/The_Geometer%27s_Sketchpad

I've written more about OpenLaszlo and Garnet:

What is OpenLaszlo, and what's it good for?

https://web.archive.org/web/20160312145555/http://donhopkins...

>Declarative Programming: Declarative programming is an elegant way of writing code that describes what to do, instead of how to do it. OpenLaszlo supports declarative programming in many ways: using XML to declare JavaScript classes, create object instances, configure them with automatic constraints, and bind them to XML datasets. Declarative programming dovetails and synergizes with other important OpenLaszlo techniques including objects, prototypes, events, constraints, data binding and instance first development.

Constraints and Prototypes in Garnet and Laszlo

https://web.archive.org/web/20160405015129/http://www.donhop...

>Garnet is an advanced user interface development environment written in Common Lisp, developed by Brad Meyers (the author of the article). I worked for Brad on the Garnet project at the CMU CS department back in 1992-3.

https://news.ycombinator.com/item?id=17360883

[...]

> because regular infix mathematics is significantly more readable than Polish notation

I agree but maybe this is simply the result of our math education

> The organization of the stuff in between really just becomes a question of standardizing function signatures and organizing modules so humans can easily navigate them.

That’s one impressive load-bearing “just” you have in there..

> functions and macros with the SAME general form: (function arg1 arg2 arg3 ...)

Macros don't have this general form in Lisp. macros just have a symbol in the prefix form, but the rest enclosed objects are not a simple list of args. The enclosed forms are arbitrary and the interpretation (parsing, transformation, ...) is done by the macro.

These can be valid macro forms:

   (infix c := a + b )

   (rule :if   (a < c and temperature > 20)
         :then set climate-control to cooling)

But those are lists of args. The macros happens to be processing the lists in a non-linear way, but they're still just lists.

Most people assume that args are restricted to Lisp syntax and that args are evaluated, like in function calls. But inside a macro form this is not the case: the macro can implement a whole new syntax and semantic -> then to say the enclosed items are 'args' is misleading: it's source code in a new sublanguage with a different syntax/semantics. Technically the macro gets called with arguments, but only as part of the process of code transformation. For the user there can be a different syntax and semantics. The enclosed code then may have a different syntax where the (operator . args) syntax is extended or no longer used.

That also means that the simple (foo . args) pattern is now no longer valid inside the macro form and analyzing the source code of a macro form can be arbitrarily complex.

> The macros happens to be processing the lists in a non-linear way

Macros are code transformers. They get code as lists and return new code as a list. This generated code then is evaluated.

Different from functions, macros are not processing normal arguments and returning an evaluation result, but they are code transformers. The resulting transformed code is then run and it returns a value.

Thus we have two (interleaved) phases of execution, instead of one:

* the code transformation phase

* the evaluation of the code

> the macro can implement a whole new syntax and semantic -> then to say the enclosed items are 'args' is misleading: it's source code in a new sublanguage with a different syntax/semantics.

Not exactly. It still needs to conform to something the Lisp Reader can absorb, i.e. something that looks like lisp. Then there are reader macros. That's where the true extensibility resides (and in addition to the new syntax elements, you'll need to reserve boundary keywords for that syntax).

> Not exactly.

That is exactly the point. Lisp syntax works differently. s-expressions are a data language and the Lisp syntax is not defined over characters, but over s-expressions.

> i.e. something that looks like lisp.

No, it would just look like s-expressions. You could define a completely different syntax and semantics on top of s-expressions: Logic (like Prolog), Postfix, ...

No one says that s-expression need to have the operator first. That's what Lisp defines as syntax. But you could write a postfix in s-expressions_

  ((3 4 +)
   9
   (pi sin)
   pi)
   2
   *)

Above is a valid s-expression, but it is not valid Lisp. As such it does not look like Lisp, but it looks like a postfix language encoded on top of s-expressions.

If a reader would reverse all expressions, then it could be executed as Lisp. You could also define a syntax where parentheses are replaced by significant indentation.

s-expressions are not Lisp syntax, they are a data syntax which is used to encode Lisp.

Historically s-expression were also defined only for data and Lisp code used m-expressions for programs and s-expressions for data.

CADR would be written similar to:

   cadr[A]=car[cdr[A]]

The function would then be called on data:

   cadr[(1,2)]

   -> 2

That's would Lisp code might look like, if it were not found out that one could als represent the code as s-expressions and that this would have interesting effects. Lisp designers tried to get away from this syntax several times.

ML switched [] and ():

similar to:

  fun cadr (l) = car ( cdr (l))

called then similar to

  cadr([1,2])

(read) reads every s-expression. Not just Lisp code. It knows nothing about the syntax of the Lisp constructs: DEFUN, LET, DEFCLASS, UNWIND-PROTECT, DOTIMES, ... When then EVAL gets called the s-expression external syntax is gone. EVAL gets Lisp code as Lisp data, not as characters or strings.

Yeah I think this is pretty much the only thing that still matters as an advantage: explicitly writing the source code as a tree, where the left part of the leaf specifies how to interpret the right part. Makes transformations pretty straightforward. Anything else is just window dressing at this point.

Too many people will end up using it to write horrible DSLs but it's the spider man thing I guess. (Please, if you think your problem is best solved by a DSL, reconsider.)

> Too many people will end up using it to write horrible DSLs but it’s the spider man thing I guess.

What is “the spider man thing”? Sounds like an interesting concept I’m not familiar with.

Also, why do you think DSLs are a bad idea? I’ve only ever heard the claim that they are a game-changer, but it’s never been clear to me why. So I’d love to hear the counter argument.

"With great power comes great responsibility."

In Spider-Man's origin story one of the first things that happens after he gains his powers is that he tries to make some money as a masked wrestler. A guy robs the box office at the wrestling venue and Spider-Man lets the guy escape because it's none of his business. The same robber later shoots and kills Spider-Man's uncle, from which Spider-Man takes the lesson expressed in the above quotation. The same lesson then colors a large fraction of the character's storylines forever after.

DSLs are great if your problem and requirements are set in stone and never change forever. Unfortunately every project I've ever had in the real world has not been the case. Every time some DSL was used to specify the logic and behavior, it ended up being an albatross around the project neck that kept it from adapting to the new requirements. It would have always been better to just use a proper programming language in the first place.

I put the DSLs into two camps. The first one, the one that I'd say is okay to use, is like Google GCL or jsonnet. This is basically what's known as a "configuration language". It has a small amount of power to let you do a bit of text processing and abstraction, but not too much. If at any point you need more than those offer you, you don't want a DSL anymore. You just want a real programming language.

Ironically, Clojure is the weakest of the Lisps when it comes to metaprogramming.

Reader macros are forbidden, and culturally, the community avoids macros. There are some good reasons to prefer functions of over macros, but one consequence is the community has less experience with metaprogramming, and uses it less.

PostScript is as truly homoiconic and polymorphic and interactive as Lisp.

And by truly homoiconic I mean not trivially and uselessly homoiconic like TCL, where "everything is a string".

And at the same time it's "JSONic" in the sense that it supports the full range of JSON data types, and is polymorphic in the sense that objects (as opposed to variables, arrays elements, and dict slots) have type and can contain objects of different types (unlike Forth, which is untyped, and is also commonly compared to PostScript because they're both stack based).

But of course PostScript was designed decades before JSON was a thing. However, the point is that Lisp S-Expressions don't directly support polymorphic dictionaries, but PostScript (and JavaScript/JSON, and Python) do.

The PostScript-based NeWS window system:

1) Used PostScript code instead of JavaScript for programming.

2) Used PostScript graphics instead of DHTML and CSS for rendering.

3) Used PostScript data instead of XML and JSON for data representation.

And PostScript (and thus NeWS) also has an interactive REPL like Lisp, to support live and exploratory programming. Imagine being able to telnet to an X11 server and create windows and draw on them interactively!

PostScript is not only homoiconic, but also point-free (or "tacit"), like Forth!

https://en.wikipedia.org/wiki/Tacit_programming#Stack-based

https://en.wikipedia.org/wiki/Talk%3AHomoiconicity#PostScrip...

https://news.ycombinator.com/item?id=18317280

>The beauty of your functional approach is that you're using PostScript code as PostScript data, thanks to the fact that PostScript is fully homoiconic, just like Lisp! So it's excellent for defining and processing domain specific languages, and it's effectively like a stack based, point free or "tacic," dynamically bound, object oriented Lisp!

The fact that PostScript code IS PostScript data, without any intermediate AST representation or reflection API, means that a PostScript data structure editor is also a code editor.

PostScript's homoiconicity and interactivity (plus the fact that PostScript is great at drawing scalable text and graphics) makes it easy to make a visual programming language and debugger interface for PostScript, with a graphical direct manipulation REPL loop that supports "direct stack manipulation" by dragging objects on and off the stack, and editing code and data by drag-and-drop and cut-and-paste.

PSIBER Space Deck Demo

https://www.youtube.com/watch?v=iuC_DDgQmsM

>Demo of the NeWS PSIBER Space Deck. Research performed under the direction of Mark Weiser and Ben Shneiderman. Developed and documented thanks to the support of John Gilmore and Julia Menapace. Developed and demonstrated by Don Hopkins. Described in "The Shape of PSIBER Space: PostScript Interactive Bug Eradication Routines".

The Shape of PSIBER Space: PostScript Interactive Bug Eradication Routines — October 1989

https://donhopkins.medium.com/the-shape-of-psiber-space-octo...

Written by Don Hopkins, October 1989. University of Maryland Human-Computer Interaction Lab, Computer Science Department, College Park, Maryland 20742.

Abstract: The PSIBER Space Deck is an interactive visual user interface to a graphical programming environment, the NeWS window system. It lets you display, manipulate, and navigate the data structures, programs, and processes living in the virtual memory space of NeWS. It is useful as a debugging tool, and as a hands on way to learn about programming in PostScript and NeWS.

PostScript Source Code Available Here:

https://www.donhopkins.com/home/pub/NeWS/litecyber/

Introduction: Cyberspace. A consensual hallucination experienced daily by billions of legitimate operators, in every nation, by children being taught mathematical concepts … A graphic representation of data abstracted from the banks of every computer in the human system. Unthinkable complexity. Lines of light ranged in the nonspace of the mind, clusters and constellations of data. Like city lights, receding …. [Gibson, Neuromancer]

The PSIBER Space Deck is a programming tool that lets you graphically display, manipulate, and navigate the many PostScript data structures, programs, and processes living in the virtual memory space of NeWS.

The Network extensible Window System (NeWS) is a multitasking object oriented PostScript programming environment. NeWS programs and data structures make up the window system kernel, the user interface toolkit, and even entire applications.

The PSIBER Space Deck is one such application, written entirely in PostScript, the result of an experiment in using a graphical programming environment to construct an interactive visual user interface to itself.

It displays views of structured data objects in overlapping windows that can be moved around on the screen, and manipulated with the mouse: you can copy and paste data structures from place to place, execute them, edit them, open up compound objects to see their internal structure, adjust the scale to shrink or magnify parts of the display, and pop up menus of other useful commands. Deep or complex data structures can be more easily grasped by applying various views to them.

[...]

Definitely, when you can directly manipulate your AST with easy, metaprogramming becomes a breeze.

Reply on news.ycombinator.com