Homoiconic Spreadsheets: What, How and Why [video] | Hacker News Reader

Homoiconic Spreadsheets: What, How and Why [video]

youtube.com

98 pointswinkywooster22 comments

intriguing presentation! So I have 1 prediction and 1 question.

prediction: even though lambdas bring "code programming" into spreadsheets, and it's a huge leap in functionality for the spreadsheet, the dominant programming mode in the spreadsheet will still be interactive programming in the "traditional" spreadsheet sense. I claim this because the mental model of code programming is different from interactive programming. That said, this will extend the reach of code programming output to the hands of more millions (who will simply copy and paste).

question: the direction of spreadsheets moving into lamdas makes as much sense as "lambdas" into spreadsheets. We've seen several new spreadsheet implementations pop up in the past years, including ones with built-in programming capability, but none of them really seem like as big of a deal as Gsheets/Excel adding lambdas. Presumably this is because spreadsheets include a ton of other features that prove its widespread usefulness even before adding things like metaprogramming. Is this about right, or did I miss a great product/language in the wild?

> We've seen several new spreadsheet implementations pop up in the past years, including ones with built-in programming capability, but none of them really seem like as big of a deal as Gsheets/Excel adding lambdas.

I think this is just because Google Sheets and Excel are used by many orders of magnitude more people than any other spreadsheet product, and so any major change in them is automatically a ‘big deal’.

True. And they are used by orders of magnitude more people because they have proven their usefulness with the masses

Stating the obvious, but it's a good reminder, because I asked the question anyway. For the mass market, it's a UX problem before it's a product problem before it's a tech problem.

Thanks for the kind words whacked_new!

I agree entirely with your prediction: spreadsheet programming is different from interactive "control flow" programming... and it should remain different! The spreadsheet model of programming is easier to grasp and incredibly flexible. I think the key is that spreadsheets are time-less & space-full, I hint at this in the talk & see also my root comment for more elaboration on this.

What's exciting to me about lambdas is that they can be used to enhance this time-less, space-based style, like in the examples I elaborate around 18m47s: the habit tracker, timesheets & a spatialized Game of Life.

I imagine/hope that in a few years we will have a culture of casually importing magical custom functions that will become polished through mass use and eventually become standard. These native functions are much safer than macros so you can be as casual with them as copy-pasting formulas. There's no permissions fuss to wade through, as there is for AppScript. (I even think there will soon be libraries of custom functions, I might even publish one myself in a couple of months.)

* * *

As to your question, I interpret it as: why are lambdas coming inside spreadsheets instead of spreadsheets becoming incorporated into programming languages, right?

I have come across many extensions of spreadsheets over the years. Spreadsheets are so ubiquitous there's many kinds of wrappers for them, many hybrids. Since the birth of spreadsheets in 1979 there has been a flood of drastic alternatives & variations that have been tried. The most famous variant so far has been pivot tables, which has become a useful, somewhat infamously intimidating feature of spreadsheets but not the revolution once expected. One of the more interesting extensions is TreeSheets https://strlen.com/treesheets/ But nothing yet has beaten the simplicity & power of a flexible grid sprinkled with formulas & references through the value rule.

To me the most important thing I learned researching this talk was that it crystallized in my mind that spreadsheets are not just a table, not just a grid. They have a key constraint, they intertwine code and data through the value rule that Kay pointed out 5 years after dynamic spreadsheets were invented (see http://worrydream.com/refs/Kay%20-%20Computer%20Software%20-... ).

So my answer to your question is that spreadsheets have been incorporated into programming languages, they have long grown scripting languages like VBA or Google AppScript. Those are massively useful for top-down extensions of the functionality of spreadsheets.

But the other direction, lambdas in spreadsheets, has been much slower & difficult, yet it offers the promise of a more bottom-up revolution in usage. It has taken us decades to evolve the model of spreadsheets towards lambdas in a way that works well with their true nature and to the ton of other features & culture they've grown over the years. (My personal favorite feature, btw, is the incredibly flexible formatting we can give to a sheet, specially conditional formatting.) As mentioned in the talk, spilled arrays are a recent, crucial step to make lambdas possible & useful.

I believe Alan drew his observation of the 'value rule' from work on ASP, Analytical Spreadsheet Package - a part of Analyst, done in the Xerox Special Information Systems Group. This system is also interesting because it used blocks (aka. closures - the object version of lambdas) as the formulas for cells as pointed out by Kurt Piersol's article in the OOPSLA '86 proceedings, https://dl.acm.org/doi/10.1145/28697.28737 .

Spreadsheet rules (or formulas) in The Analyst are kept as Smalltalk blocks. A block is a common Smaiitalk object class which allows a section of compiled code to be kept as an object. They can be passed as arguments and stored as variables. Obviously, they are an ideal choice for rule storage and execution, since they allow the rules to be directly executed by Smalltslk at compiled speeds.

The description of ASP (and the Analyst package from which it was split out as a separate product) can be found at http://www.bitsavers.org/pdf/xerox/xsis/XSIS_Smalltalk_Produ... . Of interest is the example of dropping a bitmap image into a cell and having an adjacent cell display the next generation of the game-of-life run on that initial cell. (That as opposed to having each spreadsheet cell in a region represent a cell in a game-of-life automata.)

As to homoiconicity, I wouldn't be surprised if Alan didn't at some point have a Smalltalk project window running full screen that was imbedded in a cell of a spreadsheet which was running in a Smalltalk project window within the Smalltalk environment. It's the kind of thing I saw him demonstrate as an aside during presentations as he popped out of the full screen project window which the audience had assumed was the root Smalltalk environment rather than a nested environment. What is a cell or window but a live code object running in the language environment after all?

Thanks for the reply.

> As to your question, I interpret it as: why are lambdas coming inside spreadsheets instead of spreadsheets becoming incorporated into programming languages, right?

Let me extend this. The power of the spreadsheet programming model (and work environment) is indisputable. In recent years, we've seen many interesting attempts to bring that power directly into programming tools (examples in the Clojure space: witheve, javelin, hyperfiddle; I even think witheve's first iteration was in fact a programmable spreadsheet, but I can't find it anymore). Conceptually these tools have the potential to converge to the same sweet spot, being a merge of 2 similar tenets. But the programming-driven side is converging far more slowly.

I'm interested in this because I think most programming environments these days are still too serial, and there's a lot to gain from moving into a more 2D -- spreadsheet like -- programming environment. It seems obvious, but still far away. It looks like spreadsheets will reach the sweet spot faster. But I like navigating code in Emacs more than I do jumping around cells and hitting F2 to edit.

This is truly fantastic.

Thanks! It was the first time I gave a public talk to programmer peers so I was quite nervous :)

Hi all, I'm Eli Parra & I gave this talk at ReClojure 2022 in December 3, 2022. Very happy to see this here in HN!

A bit of personal background of the talk: my work over the years has involved making internal tools for fast-moving, small organizations and that's how I fell in love with spreadsheets. They are the best canvas I know of to whip up custom, user-friendly interfaces in a couple of hours. These prototypes often then go on to be intensely used for months and even years with organic growth & minimal maintenance.

Spreadsheets are often seen with condescension by technical people & they've grown a heavy patina of drudgery over the decades. But to me they represent creative freedom & the easiest interface to computation we've yet found, so successful it's nearly invisible. I gave this talk because I'm starting to find answers to what makes them so successful and have ideas of how they might evolve.

My key points in the talk:

* Spreadsheets matter. Used by billions of people, they're the main way to go beyond being a point-and-click user and start programming. I frame spreadsheets as level 2 in a scale for interfaces that Gordon Brander imagines in this essay: https://subconscious.substack.com/p/a-kardashev-scale-for-in...

* Spreadsheets are changing. In 2022 Excel & Google Sheet got lambdas and reached level 3 in Brander's scale. The talk is a stab at what a level-4 spreadsheet might be, how you could translate QUOTE & EVAL into spreadsheets.

* Spreadsheets have an essence, what Alan Kay described as "the value rule" back in 1984: a cell can read from any other but can only write to itself. This is the founding, simplifying constraint of spreadsheets, akin to how structured programming outlawed go-to’s in code.

* Spreadsheets are time-less. The consequence of the value rule is that there is no time in spreadsheets. There are no unfolding sequences or loops or conditionals, execution happens in a subjective instant after a user change and only then.

* Spreadsheets are easy because they're time-less & space-full. They do without the trickiest part of computation (invisible unfolding time) and include a native, idealized space, the biggest aid to computation we’ve found: an interactive grid you embody & reference through cells. As opposed to ”normal" control-flow programming that goes wild with time & has abstract data structures for space.

* Homoiconicity can be practical! When translated to spreadsheets, homoiconicity looks very related to copy-paste, links, transclusion & component/instances.

To sum up: Spreadsheets are old but they've never been "finished", they've keept evolving and they merit a new look.

I am not a fan of anonymous functions. I think they are evocative of maths culture tendency to remove and make meaningless names(for example, using greek letters). Yes I know naming things is hard, but names are important, they are critical labels about the inner inner working of the machine.

Nothing wrong with functional programing, in fact it is pretty great, But I wish people would name their functions.

On the other hand, sometimes it’s better to avoid naming functions, especially if they’re small and only used once. For instance, if I’m doing something like ‘map (\(x,y) -> (x,y+1))’ to increment the second element of each item in a list (using Haskell syntax), naming the anonymous function won’t necessarily make anything clearer. Or consider this example from my own code:

    zipWith (\i m -> fmap (i,) m) [0..]

Sure, I could rewrite it as ‘zipWith prependToEachElement [0..]’, but if you know Haskell that’s not all that much shorter or clearer, especially for a function which might only be used once in the whole program.

This assumes you're writing generic, mathematical code. Maybe that's common in Haskell, and I agree that there often aren't helpful names in that situation.

But usually, if you're working with a list, it's a list of something specific, and the names should reflect that. If we're doing graphics and those (x,y) points are interpreted as points on a graph, then it's really:

  moveUpOne = (x,y) => (x,y + 1)
  map(moveUpOne, points)

Or maybe it's moveDownOne? Depends on your coordinate system.

Similarly, you're probably prepending to a list for some application-specific reason. The code should use application-specific jargon.

In Haskell you can implement moveUpOne as:

    moveUpOne = fmap (+1)

...which uses a map over a functor, in this case a tuple. But once you're familiar with functors, there's not that much benefit in naming something like that, much as you wouldn't typically define `add1 = (+1)`.

That's significantly less readable since you can't see that you're working with ordered pairs that have x and y coordinates. Don't hide the data structure.

I think it's a balancing act between labeling and mislabeling. Some things are so hard to name that naming them actually hurts. Also sometimes you don't want to pin down what it actually is; naming restricts the concept.

Wrt maths, you did maths without greek letters and with plain english, you'd have pretty ugly formulas. Greek letters, while obtrusive to newcomers, can be a lifesaver for conciseness in expression.

in the right context, greek letters were actually supposed to represent meaningful names. especially with Greek historically treated as the lingua franca for science.

the fact that most people abuse greek letters for this purpose says more about most people than about the utility of greek letters in formulae.

(but otherwise, yes, I agree with the more general point)

Have you checked the interface that Google Sheets came up for custom functions? It's in the menu of every spreadsheet in Data > Named Functions. It's pretty good and encourages not only extensive naming but documentation & examples of the function and each argument. These descriptions are then displayed piece-meal as you edit a formula, just like native documentation.

I also love good names! I hated not being able to have names in formulas and the name part of Named Functions is alone a big quality of live improvement for me.

As to lambdas, perhaps calling them anonymous functions is giving the wrong impression here. What lambdas really are is a simple, native way to create a context with local names. They can be a great boon to clarity through naming!

Spreadsheets have been historically so limited that programmers have resorted to hacky ways to have names/comments. The N function for numerical values is used sometimes for this since N of a string defaults to 0: so N("Name/comment") = 0 and can be added to a formula without changing it.

As to the cultural practice of ultra-short, likely meaningless variable names, like single letters. I do agree it's a "mathy" thing and can quickly descend into maddening opaqueness. The trick though is that sometimes it's very useful to have as compact a symbolic representation as you can manage (such a process of abbreviation gave us algebra!). Naming things is hard, perhaps one of the truly hard things in thinking/programming as the saying goes :)

You know what language is also homoiconic? Julia.

Saying that is pretty out of context. It's also incorrect. Julia has runtime code generation and hygenic macros, but it's not homoiconic. Clearly,

  x + 1

is different from

  Expr(:call, :+, :x, 1)

I tend to agree that Julia and other modern programming languages with hygenic macros aren't "homoiconic". Maybe GP is making a point about the title of this presentation? Read generously, maybe he's criticizing calling this homoiconicity by comparing it to Julia which originally claimed to be homoiconic but removed the claim from its website because of contentiousness[1]?

[1] https://groups.google.com/g/julia-users/c/iKxqn-J9frI/m/QzaS...

He could be making that comparison, but I read the comment as "I use Julia btw", and that's what I took issue with. I wrote a comment to dismiss it and save face for Julia programmers.

Reply on news.ycombinator.com