Lisp in Space
corecursive.com
corecursive.com
What would you like to see NASA do differently?
What do you think of Racket (the programming language)?
I've never used Racket but from what I can tell it's a fine system. I'm partial to Common Lisp myself, never really resonated as much with Scheme, though I do admire the elegance of the language. But if it works for you, go for it.
What do you think about the efforts of some Haskell folks to make embedded languages that would generate code for a more limited system but at the same time leverage the beefy Haskell stage as both an extensible static analyzer and an overgrown macro processor? There were a number of those (I think shaders for earlier-generation GPUs, real-time control code, and even hardware synthesis were all among the attempted targets) and I don’t know that they got any adoption at all, but this sounds similar to what you’ve described doing on less capable machines with LISP.
I don't know anything about what is happening in Haskell-land, but from your description it sounds like what they are doing is very similar. Haskell shares a lot of intellectual DNA with Lisp so it's not surprising that it would end up being used this way.
[1] https://flownet.com/gat/jpl-lisp.html, see the "Miscellaneous stories" section.
I don't know if they pursued the use of Haskell any further or even if they still use FORTH. In 2009, when I read this paper and others from them, I discovered that the Lab had hosted a by-then-defunct FORTH Users Group, not exactly an auspicious sign.
Is this design tool being actively developed or does the company want to phase it out eventually? Also sorry to ask as it may not be in this spirit of this thread but are there any open roles for a CL developer on this project?
The tool is called Meta [1] and it was originally developed at a startup called Barefoot Networks which was recently acquired by Intel [2]. Whether Intel keeps it or not is still TBD.
> Also sorry to ask as it may not be in this spirit of this thread but are there any open roles for a CL developer on this project?
Possibly. Drop me a line if you're interested.
---
[1] https://www.youtube.com/watch?v=oGQd-suLvzQ
[2] https://newsroom.intel.com/editorials/intel-acquire-barefoot...
Now, how do you see the current Lisp landscape compared to your experience back in the 80s? Where are the good opportunities for Lisp nowadays?
So I hope Lisp has a bright future, but I wouldn't be my life savings on it right now.
In my experience "space" sells to the kids more than money, fast cars and fame. So thanks for your inspiring writing about your work - I'm looking for ways to convince the dean to let me switch some courses from Python (which has jobs) to Lisp (which has excitement, space adventures and really wild things).
Did you ever try Smalltalk? Is there any area where you would prefer Smalltalk over LISP?
If you had to start from scratch, which of the two would you learn today, and why? (Let‘s assume that getting hired is not your concern, but only the ability to build powerful systems and the ability to reason about difficult problems.)
Thanks!
I think that Smalltalk deserves a little bit of opened mind here. In many respects, it is the closest thing to Lisp machines experience you can get today.
That's not quite true. In Forth it is impossible to tell syntactically where the function calls are.
> it is the closest thing to Lisp machines experience you can get today.
I think Clozure Common Lisp running on a Mac (i.e. with the IDE) is pretty close as well.
But I don't begrudge anyone their Smalltalk. It's just not a good impedance match to my brain.
I think that's the answer to every language war ever. "To my brain." If you claim "X is easier to understand", you get a war. But if you claim "for my brain, X is easier to understand", that's a lot harder to have a war about. All someone can reply is "well, my brain is different", which... OK, your brain is different. Fine. (Or someone could claim that nobody could actually have a brain such that X is easy to understand, at which point it's pretty clear who the unreasonable zealot is.)
In fact, I think this is also the answer to the editor wars. It may even be the answer to the political wars.
To me, Lisp makes every other language I know feel like BASIC in the sense that the distinguishing features of the language feel to me like they are designed to appeal to ignorance rather than to empower.
"IMHO" and "objectively" are in fairly stark opposition.
I agree, but the scope of your assessments will be very limited, and also you end up writing tautologies (if your objective quality metric is A, then you've redefined "better" to mean exactly A). In particular, the scope of your assessments will be far more limited than the example you give (words such as "easy," "simple," etc. are unlikely to enter the statement without some rigorous definition, but therefore also highly restrictive, underlying it).
> For example, C is objectively better than Brainfuck if your goal is to write software easily.
I would not take a bet, that if you presented Brainfuck and C to all > 7 billion living humans, there would not be at least one person who found it easier to write software in Brainfuck than in C. Certainly I could imagine a hypothetical human for which it would not be true.
To be fair, this is true of Common Lisp as well, macros aren't functions.
Forth doesn't really have functions, it has words, they're broadly useful like functions but are a differently-literal sort of memory object from a lisp function defined with cons cells.
So you can't know where the function calls are, there is no such animal, but you do know where the word boundaries are, it's the whitespace, and in Forth that's what you care about.
That's true, and it is arguable that Lisp could be improved by making macro invocations have a distinguished syntax.
But in practice it's usually pretty clear what is a macro and what is a function because of naming conventions, and because macros commonly admit syntax that would be illegal in a function call e.g. (let ((x 1)) ...) But however bad this problem is in Lisp, it's much worse in Forth.
> Forth doesn't really have functions
That is debatable. If I write 1 2 + 3 * then I think of the + and the * as function invocations that take their arguments off the stack and leave their results on the stack. It's true that this is not exactly the same as a function call in Lisp, but it's similar. In any case, if I write:
a b c d e f g h i j
there is absolutely no way to tell what that is going to do without knowing what every letter means. By way of contrast, if I write:
(a (b c d) (e f) (g h (i j)))
then it's a pretty good bet that a is a function that takes three arguments, b and g are functions that takes two arguments, e and i are functions that take one argument, and c d f h and j are variables.
That's what I mean by Forth words not really being functions, they're subroutines for sure, and you can do some surprisingly powerful metaprogramming given that what you have is a glorified linked list, something a little more powerful than GOSUB, and two stacks, but functions?
When you compare it with a visually similar concatenative language like Factor or Joy, which inarguably have functions, Forth words look pretty different.
Agreed.
a b c d e f g h i j
(a (b c d) (e f) (g h (i j)))
Although I do like Lisp generics - I did a bunch of programming in rscheme many years ago, and really liked it.
Why? Seriously, why is that something you want? Why do you want sending a message to be syntactically and semantically distinct from calling a function?
> The problem people get into with OO (particularly Java, C#, C++) is thinking in terms of method-function calls rather than message sends.
Well, IMHO the problem people get into with OO is thinking that there is something special about sending a message that is different from calling a function. Sending a message is an implementation technique, not a semantically distinct action that should be exposed in the language semantics, and certainly not in the syntax.
In Java it is pretty much just a function call, because the code in the callee has the ability to wreak all kinds of havoc in real Java programs, so the pattern of "messages" is quite imperfectly related to the authorities you're bounding.
More at http://habitatchronicles.com/2017/05/what-are-capabilities/
Then there are Proxy objects, reflection, and invokedynamic as well, for those "doesNotUnderstand:" scenarios.
Sure. But why do you want that difference to manifest itself in the syntax of your language? What is the benefit of writing, say:
[probe message]
over
send_message(probe, message)
?
> OOP approach makes this assumption implicit for everything.
And that is exactly the problem IMHO because this model is not appropriate for everything. It makes sense when you are dealing with actual physical objects like a Mars probe, but that is rare. Much more commonly you are dealing with abstract objects. If I want to add two numbers A and B, who do I send the message to? A? B? The adder in the ALU? As a programmer, 99.9% of the time I don't care how the operation gets carried out. I just want to write (sum a b) or a+b and let the compiler sort out the details.
probe message
without any brackets, and you can write
a+b
which is the message + send to a with argument b. You just have message passing syntax for everything (Smalltalk has minimal syntax comparable to Common Lisp in size). IMHO it hit the sweet spot of simplicity/readability/expressiveness.
But you cannot judge Smalltalk only from the perspective of its syntax. You need to take into account the context in which it is used. The environment built around it. Then everything starts to make much more sense and forms a vital well balanced system. It is Lisp but different, and I strongly recommend downloading Pharo and to try to figure out how.
I stronlgy admire you; I read "Lisping at JPL" at least dozen times over the years. I really like Lisp, Forth and Smalltalk, and I know that knowing each of them well is worth every penny, even if it may not be obvious on first sight.
Yeah, I get that. What I don't get is why I should care that "a+b" means "send the message + to a with argument b" rather than just "add a and b". I see no benefit of the first formulation over the second.
(And thanks for the kind words!)
Yet I never got the point of arguing about implementation details regarding virtual method dispatch versus message passing.
Specially when many OOP GUI frameworks combine both approaches.
It really comes down to fetishism, which is why I don't use either of them. I am not a Fetish-Oriented Programming enthusiast.
It is also why I am not all in on Rust. Memory safety is pretty important, but it is far from the only important thing in programming. The amount of attention Rust insists I devote to it is attention I don't have available for the other things that, frankly, deserve it more.
Considering that in the last 10 years I have spent strictly less time debugging memory usage errors than preparing compiler bug reports against Gcc, putting memory correctness guarantees front and center ahead of all else seems to me a mis-allocation of my extremely scarce attention.
Attention is by far the scarcest resource available to any programmer. Allocating and applying attention where it is most needed is the central problem of programming. A language that overrules your judgment as to where your attention must be thus interferes with the core business of programming.
C++ does demand just a little more attention to memory management than some other languages, but less all the time. It pays that back by eliminating most taxes on performance, a swap I agree to.
With regards to LFE, I don't really see the point. But that might just be because of the kinds of coding I do. I've never written high performance massively parallel code. Message passing might have a benefit there (that is what the big win with Erlang is supposed to be) but I'm generally skeptical of designing the language semantics around the needs of the compiler rather than the programmer.
In particular, I don't see what Erlang could possibly do that could not be done by a CL compiler that noticed when a method was dispatching only on the first argument. The semantics of that are equivalent to the semantics of message passing, and so the code that such a compiler emits could be identical in both cases. But I don't actually know Erlang so it's possible I'm missing something.
wrt LFE specifically, Robert Virding, one of the founders of Erlang, really likes languages and Lisp in particular so he wrote one for the BEAM. It just has some special accommodations to be able to send/receive (and I think pattern recognition too).
It is ported to every major flavour of OS.
I don't know what 'back end' means in this context.
You can also compile a high level language down to machine code that runs on actual hardware, like an x86 or an ARM. For languages that run on more than on processor, the compilation process usually consists of an architecture-independent phase which produces some sort of intermediate representation (which may be a byte code, or it might be something else, like LLVM) and then an architecture-specific pass that transforms the intermediate representation into machine code for the target architecture. The code that implements that second pass is called the "back end."
I have no idea whether Erlang compiles to native code or byte code.
In addition to all that, there can also be a run-time environment that is required to run the resulting code. For byte code, this environment necessarily includes an interpreter for the byte code, and might also include other things. For native code this environment might include things like a garbage collector or a standard library that provides an interface to an operating system or something like that.
There are multiple passes, the last of which is byte code although at one point, native code was (is?) an option with HiPE (High Performance Erlang). HiPE seems to have been passed over by the development team in favour of JIT.
There also used to be an implementation of the BEAM VM directly on Xen: https://github.com/cloudozer/ling / https://web.archive.org/web/20190507184436/https://erlangonx...
I wish people would stop saying this. Writing concurrent systems in BEAM requires thought if you want to avoid deadlocks, races, bottlenecks, and performance problems. It also can be a bit tricky because processes serve many roles at once: fault isolation, GC isolation, the unit of concurrency, and the unit of synchronization.
Yes, a process can unleash concurrency and parallelization. But you also use processes to do things synchronously since a process goes through its mailbox one message at a time. You can accidentally bottleneck your system because of this.
Regardless, it does make it easier to write certain types of programs.
A friend wrote on the topic here: https://dorgan.netlify.app/posts/2021/04/the_elixir_ast/
(This is Darius. I never really learned your system with related goals for robots -- as you know, I didn't stick around at JPL. About 10 years later I did a bit of Erlang for Yahoo.)
Do you think anything survived that would shed more light in its workings? Not even talking about the code (although that would have been wonderful, and maybe it should be provided - taxpayer funded and whatnot).
The first link is the one I keep coming back to, and I keep finding interesting idea. But it didn't occur to me to follow the people involved.
Thanks for the references!
He used DSL's written in LISP and compiled to run on embedded devices and did self-driving tech all back in the 80s. Also, debugging software running 150 million miles away is something I'm sad yet relieved I'll never get to do.
Also the photos he dug up and scanned are neat to see.
If they are using compiled C programs it seems like it would be very hard to make changes. Is the key just to not need to make any changes?
Or is this a case of Greenspun's tenth rule, where they've built REPL like functionality on-top of C?
And they don't typically even have a REPL, as doing brain surgery on a satellite (that's already in a bad way if you're debugging something) is rarely the best option. So lots of telemetry logging, with 'feature flags' to turn on different logs. Then a change and code push once root cause is found, or a new build with new telemetry gathering if it's still not clear.
I'm sure just about every model for code structure has someone doing it though.
It is findable online, but he published a book "Sunburst and Luminary" (Fort Point Press, 2019) about the whole process of getting the moon landing code ready in time to use, and the hack.
As I understand it (I just got the book, haven't read it yet) the Apollo Guidance Computer, one each in the Command Module and the LEM, was programmed by Margaret Hamilton (inventor of Software Engineering as we know it today) with a real-time executive and interpreter emulating a saner machine, and Eyles coded the landing to the interpreter. Because it was interpreted code, it was patchable, and he came up with a patch on the fly that the astronauts punched into the AGC by hand, and saved the mission.
Margaret Hamilton's real-time executive itself saved the day when Apollo 11 crew left some extra stuff turned on, by accident, that the system had not been tested with and that burned excess CPU cycles during the landing. When it trapped a scheduling failure, it checkpointed important state and was able to resume the important tasks where they left off. That happened several times during the landing.
https://www.goodreads.com/book/show/39294745-sunburst-and-lu...
I ordered this one directly from https://www.sunburstandluminary.com/SLhome.html and it showed up very quickly with a minimum of fuss.
If you were to handle it today, how would you do it differently? Any advice to an engineer to learn the art of selling more powerful tech against a lesser one when the latter is more popular?
More projects can be found at: https://www.libhunt.com/topic/common-lisp