Hacker News Reader

Comment by sph | Hacker News Reader

ADDENDUM: I love Lisp but I want to clarify that its secret sauce is that it enables exploratory programming: programming from inside the system.

While most languages are akin to preparing a detailed blueprint and executing it (then stop to update the blueprint and try again), exploratory programming is like sculpture. You have an unformed mass full of unrealised potential, and you slowly carve it to shape.

The ADHD/Bipolar mind of the article isn't great at making plans, it's great at following its gut instinct, making shit up step by step.

Lisp enables this flow, but also Forth, Smalltalk, Self, etc., which I've been deeply obsessed with lately. Sadly, all we have today on the mainstream are basic REPLs which are a harmless, defanged version of these old environments promoting creativity and flow state.

View on HN

"The ADHD/Bipolar mind of the article isn't great at making plans, it's great at following its gut instinct, making shit up step by step."

Which is why I do best with the command line, piping outputs and rapidly iterating to get the output I want.

And why I struggled so much writing Python in a larger, more complex script/app. I wanted to constantly run and rerun the smaller parts to get each one just right. Unfortunately that doesn't work great when you're interacting with a half dozen different APIs that take minutes to run each time. Until I realized what was slowing me down, I was just getting more and frustrated.

Now I am at least aware of the issue. Part and parcel of growing from sysadmin to DevOps to coder, I guess? Now I know to look for shims or stubs that help me. Now I can write tests that isolate and verify, rather than run the whole damn thing end to end.

It helps me to think of objects as those command line tools. Their method parameters are the stdin that get passed through pipes. I apologize that this isn't particularly profound, but sometimes epiphanies are made from smaller realizations.

Maybe I should have tried writing part of my most recent script in Lisp? Just to see if it resonated like described in this thread?

> I wanted to constantly run and rerun the smaller parts to get each one just right.

> Maybe I should have tried writing part of my most recent script in Lisp?

90% of Lisp programming is selecting an expression in your editor and pressing C-x C-e to execute it. And continue until you are satisfied it operates as you want, then integrate it in the larger system.

> I wanted to constantly run and rerun the smaller parts to get each one just right.

One one hand, as you alluded too, I've found unit tests are a great solution to this. Kill two birds with one stone - rapidly iterate on a chunk of code and come out of it with solid test coverage.

It's too bad that's not very feasible in gamedev most of the time with so much going on in the world, but over time I think that environment improved my skills to where it's less of a problem.

what about gnumeric, observablehq, jupyter, the unix shell, and the browser console (firebug and its clones)

admittedly it's hard to save the state of the page after you hack on it in the browser console, but jupyter doesn't really have that problem

> programming from inside the system.

Can you elaborate on that? I think you are saying that REPLs in other languages don't do what you can do in Lisp, but I don't think I know enough to fully understand why.

Imagine this:

You start a fresh REPL session with a vague idea of something you want to build. You have good tooling that enables you to write code in your favourite editor and pipe it to the REPL through a socket. You write a couple of functions and send them to the REPL. You call them a few times with some test arguments, maybe define a couple global variables while testing, and realize one of the functions doesn't quite behave the way you want. You tweak the code and send it again to the REPL. The function is redefined and works properly now.

You're happy with what you have for now, so you save the entire current environment of the REPL, functions and global variables and all, into a file (called an image) on your disk. The next day, you come up with a great new feature so you load up the image and continue exactly where you left off yesterday, with all your functions and even global variables as they were defined previously. You add your new feature, all while testing and tweaking your functions interactively, and then you save the new environment into an updated image. Any other code that loads that image will be able to take advantage of your updated functions and variables now.

The power of lisp is not just the fact that there's a REPL — a lot of languages have that now. It's the ability to save and reload the entire current state of the REPL (interactively or programmatically), which enables the powerful interactive development that so many lispers, including myself, rave about.

What happens when you want to share the thing you built with somebody else? If I type a defun into a REPL, is there a good way to get source code back out again?

Because there's a problem I've run into several times when writing Lisp (I use SLIME): I decide to refactor some stuff, so I rewrite some stuff, maybe remove a function or two from the source as part of that. Things seem to be working fine. I save my .lisp files and then at some point end up restarting the Lisp process (often because McCLIM can be kind of flaky and lose its connection to the X server). When I go to load my code back in, I get errors, because oops I was actually still calling that deleted function somewhere -- it "worked" before because the old function was still hanging out in the image.

When I develop in a compiled language, I just accept that I'm going to be re-building the entire thing from scratch (equivalent to restarting the lisp process & re-loading the source files) every time I want to try a change; if I broke something, I'll find out the next time I run make/go build/whatever. When I'm developing in Lisp, and I'll freely accept that my Lisp dev flow probably isn't ideal, it's so easy to screw up but be unaware of it for hours if not days: I write some code, I'm happy, I close Emacs, then a week later I come back and try to load my package and it's just broken.

I could solve it for myself by just using images: the "deleted" function would persist in the image, and every time I started working again it would be there. But if I ever want to let somebody look at the code, hack on it, make changes, what's that look like?

> When I develop in a compiled language

"batch compiled from scratch"

OTOH, most usage of Common Lisp is compiled, often by an incremental in-image compiler

> because oops I was actually still calling that deleted function somewhere

Typically Lisp IDEs have features like Edit Callers and/or List Callers. I would

1) "Edit Callers" -> this walks me through all functions which use this function, I change the functions and I can edit/compile them

2) When I'm done, I use "Undefine Function", which then undefines the function from the running Lisp.

> "Undefine Function"

That's the key; if you delete a function make sure to undefine it so it's not sticking around in the image.

In slime you can use `C-c C-u` which is bound to FMAKUNBOUND.

> If I type a defun into a REPL, is there a good way to get source code back out again?

this used to be the normal way of programming in lisp repls (like in 01980) but maybe this is good enough

    * (defun tri (n) (loop for i from 1 to n summing n))
    TRI
    * (describe 'tri)
    COMMON-LISP-USER::TRI
      [symbol]

    TRI names a compiled function:
      Lambda-list: (N)
      Derived type: (FUNCTION (T) (VALUES NUMBER &OPTIONAL))
      Source form:
        (LAMBDA (N)
          (BLOCK TRI
            (LOOP FOR I FROM 1 TO N
                  SUMMING N)))

> When I go to load my code back in, I get errors, because oops I was actually still calling that deleted function somewhere -- it "worked" before because the old function was still hanging out in the image.

so a thing you can do in that case is to delete it from the image at the same time

well, you can't really delete it from the image, but you can remove the function binding of the symbol, and since normally in common lisp implementations function calls indirect through the symbol (which is why you can replace the definition with a newer one) deleting the function binding is enough

    * (defun tri0 (n) (tri (1+ n)))
    TRI0
    * (tri0 4)
    25
    * (tri0 5)
    36
    * (fmakunbound 'tri)
    TRI
    * (tri0 5)

    debugger invoked on a UNDEFINED-FUNCTION @52A00674 in thread
    #<THREAD "main thread" RUNNING {1001348003}>:
      The function COMMON-LISP-USER::TRI is undefined.

    Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

    restarts (invokable by number or by possibly-abbreviated name):
      0: [CONTINUE      ] Retry calling TRI.
      1: [USE-VALUE     ] Call specified function.
      2: [RETURN-VALUE  ] Return specified values.
      3: [RETURN-NOTHING] Return zero values.
      4: [ABORT         ] Exit debugger, returning to top level.

    ("undefined function" 6)
    0]

as rainer pointed out, slime has a command for this (i barely use cl myself, you know much more than i do)

I work in a slightly different way by always saving my code in emacs and loading it fresh each session (though I have gone a couple of weeks with a live image/session on my laptop). I am fortunate that my 3D system [0] compiles and loads in a few seconds.

I also use the REPL for quick tests, and evaluate new code by writing it in emacs and compiling it one function at a time (C-c C-c).

Finally, I keep files of tutorial/demo/test cases which work a bit like a notebook, allowing me to quickly verify that existing 3D scenes still work.

[0] https://github.com/kaveh808/kons-9

About 15 or 20 years ago I had a work flow using SBCL Common Lisp where I saved images and always restarted a new work session where I left off. Easy to do.

I was motivated to do this because I had a ton of linguistic and text data in my environment which back then took a little while to load. Now, I organize everything I do into small or sometimes tiny Quicklisp projects/libraries and that is much more convenient for me.

If you want to really get into image based programming, the open source Pharo Smalltalk is very good.

This is the kind of thing that should be included in language tutorials!

It particularly resonates with me because I think so many project teams neglect the Developer Experience. In my opinion DevOps should be more about making coding, testing and deploying as frictionless as possible.

when this term first came out, it really seemed like that was the point

now it just seems like a hundred children you have to pacify and get to bed before you can even start working

That is very helpful. Thank you. Most of my exposure to Lisp has been Scheme and Racket and minimal at that. Are these images you describe something that is part of a particular distribution of Common Lisp?

Images are pretty routine amongst most of them (though not all).

In the end what you have is a base image, and then a set of code and instructions to transform that image into your runtime image, then you distribute that image as your "application".

I have never needed to do this, as all of my work has been utilitarian in nature, rather than some kind of monolith distributed to others. I simply load my code on to the stock image and go from there.

But even then, for folks that do that, they may well load up the stock image with useful and common, personal, utilities, and snapshot that. Their "application" is their essential working environment.

Smalltalk does, and Forth can work from this concept of an image.

Schemes tend to not be this way. They are interactive, but the image concept is not as ingrained in the Scheme world. Instead, it's more like doing development in something like Bash -- an interactive environment to be sure, but one that always resets to the baseline state and must be reloaded when started. There are certainly Schemes that can do this, it's just not as routine.

Reply on news.ycombinator.com