Overview and Introduction to Lisp (1986) [video] | Hacker News Reader

Overview and Introduction to Lisp (1986) [video]

youtube.com

156 pointstosh61 comments

A really moving and inspirational quote from the opening of SICP:

"A computational process is indeed much like a sorcerer's idea of a spirit. It cannot be seen or touched. It is not composed of matter at all. However, it is very real. It can perform intellectual work. It can answer questions. It can affect the world by disbursing money at a bank or by controlling a robot arm in a factory. The programs we use to conjure processes are like a sorcerer's spells. They are carefully composed from symbolic expressions in arcane and esoteric programming languages that prescribe the tasks we want our processes to perform."

More than anything else, that paragraph is what kept me coming back when I first started learning to program and struggled mightily. I lugged that huge book around in my book bag for months, trying and failing to fully grok one of the earlier chapters, until I found a professor who was willing to help me with it. Ultimately we (the professor and I) were able to do a one semester independent study focusing on the material from SICP.

I used to read and re-read that paragraph. I read it aloud to my parents and to girlfriends, trying to find somebody to share my awe.

Sometimes it is hard to find others who share an appreciation of the beauty of mathemtics or computing. People look at me funny when I say programming is every bit as much an art as writing a poem. Almost any human activity can be an art.

> writing a poem

On that analogy, Lisp is like French. Other languages are like German.

(And before you downvote me please note: I am a native German speaker.)

That's funny because I am a native French speaker and I find German to be a very pleasing language to speak out loud. The sonorities are oddly satisfying to me.

Please note that I am nowhere near fluent in German. It just feels great, especially the way words can agglutinate to form comically long words, while still transporting meaning.

I don't dispute any of that. But French is still a better language for writing poetry (IMHO).

I see. I kinda mixed up poetry and beauty.

Les mathématiciens sont comme les Français: quoiqu'on leur dise, ils le traduisent dans leur propre langue, et cela devient alors quelque chose de complètement différent. — Johann Wolfgang von Goethe

Incorrect; a computational process is made of matter/energy.

It has an conceptual interpretation of what it's doing; what it represents. That is what is intangible.

This series is brilliant. Why don't they teach computer science this way any more? I was so disappointed to learn that MIT switched from Scheme to Python in Programming 101.

Theoretically, all the same concepts could be taught in Python as well, but I doubt they are doing that. Many of the more mind-bending things that are made relatively easy in Scheme due to the uniform syntax and small core actually become much more complex to do for Python (e.g. writing a self-interpreter).

The reason why is because in the 80s it was useful to build up programs from scratch. So you could build an introductory course around that and even use an arcane language like Scheme, which the students are bound to almost never encounter in real-world usage, to illustrate the principles. These days, to teach someone real-world software development, you have to show them how to find and use a library that does what they want, or maintain an already-existing code base -- use cases for which Python is a much better fit. Building a program up from scratch is almost never done.

> an arcane language like Scheme, which the students are bound to almost never encounter in real-world usage

Posted on a site running on MzScheme.

>> an arcane language like Scheme, which the students are bound to almost never encounter in real-world usage

>Posted on a site running on MzScheme.

Posted on a site running MzScheme, that probably tens, if not hundreds of thousands of people, read often, get useful info from, that is of help to them in the real world, and that many people get jobs from.

This shift of focus from fundamental computer science to "real-world software development" training seems to be short-sighted and more appropriate for a vocational school or a coder bootcamp rather than a place like MIT.

In the 1980s, people used libraries and maintained already existing code bases in "real-world software development", too.

Computer science education isn't just vocational training for software development gruntwork, though.

Yes. As Perlis put it:

   54. Beware of the Turing tar-pit
       in which everything is possible 
       but nothing of interest is easy.

Unlike Scheme, Python requires extensive research of the documentation in order to get the correct syntax and semantics even for the simple code in SICP. It's not just simple facts like operator precedence, it's the conceptual baggage of statements versus expressions. It's the one true way of iteration rather than recursion. It's the arbitrary distinctions of a core language, built in functions, a standard library and PiP. And Python has objects just because. But everything isn't an object.

SICP was an introductory programming class (though obviously being used at MIT allowed certain assumptions).

"[...] Costanza asked Sussman why MIT had switched away from Scheme for their introductory programming course, 6.001. This was a gem. He said that the reason that happened was because engineering in 1980 was not what it was in the mid-90s or in 2000. In 1980, good programmers spent a lot of time thinking, and then produced spare code that they thought should work. Code ran close to the metal, even Scheme -- it was understandable all the way down. Like a resistor, where you could read the bands and know the power rating and the tolerance and the resistance and V=IR and that's all there was to know. 6.001 had been conceived to teach engineers how to take small parts that they understood entirely and use simple techniques to compose them into larger things that do what you want.

But programming now isn't so much like that, said Sussman. Nowadays you muck around with incomprehensible or nonexistent man pages for software you don't know who wrote. You have to do basic science on your libraries to see how they work, trying out different inputs and seeing how the code reacts. This is a fundamentally different job, and it needed a different course. [...]"

(From http://wingolog.org/archives/2009/03/24/international-lisp-c...)

Surely the best engineers writing code for cars and aeroplanes are not programming like this? MIT is teaching people to be web devs.

One of the most common complaints about MIT CS (from outsiders and hiring managers) is that it’s not practical enough. There’s very little “applied” industry CRUD-type programming in the curriculum. I think this is a good thing, but also lament the changes in 6.001.

If courses were less expensive, they could afford to be less practical. Maybe this wouldn't quiet the hiring managers, but it's easier to tell someone to screw off when what you're selling isn't a huge investment which has been built up to be vitally important to justify that pricetag.

I don't think MIT EECS department has much to fear in the way of selling itself. MIT turns away almost 12 undergrad students for every 1 it admits.

You're wrong. The only real exception is the code in airplanes, failures for which can cause loss-of-life (e.g. Jet ECUs, fly-by-wire). Even then, in some cases, the solution is not correct software, but a hardware fail-safe.

Everything else is pretty much programmed like this. I've seen things you'd likely describe as "deeply embedded" running a CORBA server so that Java written by one company can communicate with C++ written by another company, integrated by a third company that doesn't have the source to either.

Some automotive software is starting to show some trends towards the airplane approach, but the big exception among safety-critical parts is self-driving cars. Many of them even use ML algorithms for which it is non-trivial to even determine why a particular error was made.

Cost is a big driver of this. Given equally competent programmers, creating formally correct software is 1-2 orders of magnitude at low levels of complexity. Current software-engineering indicates that the difference increases as complexity increases. You can easily end up spending 100x as much for software that has 1% as much complexity.

Much of the time, features drive sales, and sales drive revenue, so the less rigorous software ends up with both higher budget and lower costs.

My impression is that the car industry doesn't invest in good programmers, they invest in controls and stringent procedures/conventions along with a conservative mindset. Stuff like tools to verify you don't use the most dangerous features of C.

>This series is brilliant. Why don't they teach computer science this way any more? I was so disappointed to learn that MIT switched from Scheme to Python in Programming 101.

Does anyone know why they did that? I did read about it around the time, but do not remember reading about the reasons. As a Pythonista, and one interested in Lisp, I'd like to know.

Update: Okay, I was going sequentially and did not at first see a few answers (sibling comments) below. Still, if anyone else has any insights, interested to know more.

It's ridiculous. My university tried to switch the CS program from C/C++ ( OS, networking, etc ) and Scheme ( functional, programming languages, etc ) to java and there was a mass movement against it and we kept the "traditional" languages. I don't know why there is such a movement in academia to dumb down everything.

One reason might be because their (academia's) customers demand a return on investment. And that return is difficult to realize when Recruiter Joe is screening you for a job and asking how many years of "The Java" you've had.

Don't get me wrong, this is in no way an argument against what you said. I'm just offering up a possible answer.

Their demands should be fulfilled by vocational schools, not universities.

The entire series of 20 lectures (uploaded by MIT) is available here: https://www.youtube.com/watch?v=2Op3QLzMgSY&list=PLE18841CAB...

I love the direct start on why "Computer Science" and "Geometry" are not ideal at conveying what they stand for.

I love that too. It's a wonderful teaching style where you get another piece of information for free along with the primary lesson. It makes it stick so much better too because you have more connections in your brain. It's similar to how I succesfully learnt French grammar which was to learn English grammar at the same time. I came out with twice the knowledge and something which is much more than the sum of its parts.

My First CS class at Berkeley (CS61A) was the last semester (Fall 2010, my first semester of college) where it was taught it in Scheme.

After that they apparently switched the class to python.

My professor was Brian Harvey whose doctoral advisor at MIT was one of the authors of the SICP book we used for the class.

It was a good book. One of the few CS textbooks I actually read. Many CS books are pretty badly written imo.

But there are a few that I consider very well written. This book was one of them.

As a side note,

Scheme and parenthesis hell when writing code on the midterms....

Also car caar cdr cddr etc etc (you can theoretically nest as deeply as needed)

hahaha that class...

> Scheme and parenthesis hell when writing code on the midterms....

Writing code on paper sucks in many (all?) languages. At least the ones I have tried. Python might actually have an advantage on that front, Haskell, too.

Yeah I agree with you.

But scheme especially when you have code like this:

(letrec ((even? (lambda (n) (if (zero? n) #t (odd? (- n 1))))) (odd? (lambda (n) (if (zero? n) #f (even? (- n 1)))))) (even? 88))

The level of nesting can get rediculous.

Writing that on a midterm is harder than Python or java.

I’d say Python is nicest when writing out code.

For a person who loves Scala, Haskell and Python and have heard a lot about LISP (and even uses a DIY LISP notation to serialize some stuff in some projects) yet didnt't have time to actually study LISP yet... In what parts can LISP or Scheme be better than Haskell as a programming language from the practical point of view? I.e. why should I choose to code a program in LISP rather than in Haskell or some other functional language?

I have been programming Lisp languages since about 1980, and in the last six years have become a Haskell enthusiast (but still learning). For what it’s worth, my opinion is that Haskell is a Lisp. When I am writing pure Haskell code (algorithms, not touching the external world) working in Emacs, it feels like my workflow is very similar to using Common Lisp or Scheme. If you like Haskell, then you have found a good home.

You should check the paper "Why calculating is better than scheming", from Phil Wadler.

i have and i'm still not seeing why calculating is better than scheming. it also lays into lack of lazy evaluation, but haven't we determined over time in haskell that lazy by default has it's own tradeoffs?

Could you maybe break it all down for us ? thanks

When you want there to be a lot of moving parts that are easy to swap out and change and develop as the end user, while using it, pick Lisp.

When you want the design to be fixed and predictable and easy to develop for the developer, before the program runs, pick Haskell.

If you want to compare Lisps to Haskell I would think of "not so strictness" in functional programming or programming paradigm. Lisp is not only functional, it has much more to offer and that is in some situations more suitable. Choose the right tool, etc. etc.

If you like Lisp for specific tasks then exactly this would be a good reason to do so.

Lisp trades some of the strictness and static typing of Haskell and Scala, and the convenience of Python I guess; for more raw power. You won't really have a clue what macros are capable of before you see it yourself, I wholeheartedly recommend using Let Over Lambda (https://letoverlambda.com) as a guide. The other side of the coin is that you'll spend the rest of your days waiting for mainstream languages to catch up, but that's true of Haskell as well to some extent.

Thanks for the pointer, just ordered a copy.

Note that let over lambda is ... let's say controversial among lispers. I think it is very good at teaching how to write macros, but very bad at teaching what macros one ought to write. It's a bit like that bit about making ruby accept bare words in Wat[1]. The fact that lisp lets you do the things that LoL presents is fine, learning how to do this is good. If you actually do this in production code, that's bad.

1: https://www.destroyallsoftware.com/talks/wat (45 seconds in)

I don't know about "won't have a clue what macros are capable of before..." -- there are quite a few use cases for macros which can be described easily to any programmer, even those who never used Lisp. I would like to believe it would suffice to say "macros in Lisp allow you to fully extend the the language with new primitives and abstractions in a way that no other language does", but examples are always helpful.

Lisp has a macro called setf, which takes any call which gets a value -- whether from a local function variable, or a class, to an array or hash table entry -- and converts it to the corresponding setter call. (If you define your own value places/types, there are also functions to make setf aware of how they work, so this isn't restricted to just some defined set of features in the language. Anything that holds a value in Lisp can be "setf" to a new value.)

There are a couple of other "big" Lisp macros in the language standard like with-open-file (which ensures a file is closed once control flow exits the body), loop (does pretty much what it says on the tin: provides whatever looping method you might want), or handler-case (try-catch style error handling; yes, Lisp allows for other methods).

Or how about the example I recently saw in a cryptography library. Since a lot of cryptographic algorithms use integer math modulo some giant number, this library defined a macro "mod-n" which took a base and some Lisp code, and re-wrote the code so the common integer math functions (addition, multiplication, etc.) would only return values "modulo {base}" --that is, return only values between 0 (inclusive) and base (exclusive). This meant their modulo math code looked (nearly) the same as normal, just standard arithmetic functions. This wasn't a very large macro either: only around 10 lines long (and about half of that being the list of operator replacements).

Practically I'd say Interactive Development and Metaprogramming facilities would be the top two reasons.

Interactive Development let you iterate ideas really fast, and allows an ease of exploration that's refreshing and can really help to learn how to use a library, an API, etc.

Metaprogramming lets you add whatever functionality you want. It can help reduce boilerplate code down to a minimum.

Those are the two features where I think it distinguishes itself.

My journey was from Python to (Common) Lisp to Haskell (which I decided to go with instead of Clojure). I got (almost instantly) sold on Haskell when I learned how to interpret the same code in different ways (http://www.haskellforall.com/2012/06/you-could-have-invented...) and that the types do not get in the way in Haskell as much as they do in C#/Java whatever.

I think Lisp is interesting, but I doubt you will find many parts that are "better" (more powerful or succinct) than Haskell.

Lisp is primarily about freedom that macros give you. However, the two huge use cases for macros, selective evaluation, and representing code as data (different interpretations of the same code), can be naturally done in Haskell. There are some other minority use cases for macros, but in my current view they do not justify the cost of losing the type system.

I now think that macros are an evolutionary dead end. They are powerful, but they are hard to integrate with the type system. It's kind of surprising that progress (IMHO) in programming comes from (selective) restriction of freedom. And that's why Lispers will probably strongly disagree with me...

Anyway, if you like Python, you might like Lisp. The core language is more elegant, and much more powerful, but the libraries are generally less polished.

Somebody recommended Let Over Lambda to show what macros can do, I don't disagree but I would recommend reading On Lisp first, it covers more traditional uses of macros.

Lisp and Smalltalk are the precursors of modern IDEs.

Many of the ideas presented by Bret Victor can be seen on those environments

Allegro Common Lisp and LispWorks are the surviving examples how they used to be.

The language malleability of Common Lisp is quite nice. If you like Haskell, you will miss the types. SBCL (the most popular open-source common lisp implementation) does what is quite a lot of compile-time type checking and inference for a lisp, but very little compared to Haskell.

Common Lisp has found a very high local-maxima for macros. If you knew all of common lisp except defmacro, I could explain how it works to you in about 3 sentences. This is an absurd lever for how much power it gives you. This is also where many "lisp-in-foo" implementations fall down.

They (rarely) use the scheme macro system (which has a very complicated algorithm behind it) or more commonly implement the common lisp system, but without the features of common lisp that make it reliable. A major exception to this rule is Clojure.

I'm not a huge clojure fan, but every single place in which it differs from common lisp, it is immediately apparent that Rich understood why it was done in common lisp, and has very good reasons for doing it differently. It's one of the very few lisp branches that I can't find Chesterton's fences[1] in. I don't think this is because there's anything wrong with lisp, but rather that it's very very easy to write a any lisp dialect, which means you get a lot more bad ones. You need to do a lot more work to write a bad

1: https://en.wikipedia.org/wiki/G._K._Chesterton#Chesterton's_...

Thank you for the link to Chesterton's fence! What a wonderful way of framing reformation/reconstruction/refactoring.

I do love that story, it really sums up one bit of knowledge that older and wiser people tried to impart to me.

Anything that takes work required some form of motivation. It is very likely that the person that did the thing you think to be wrongheaded was a lot more like you than different from you.

The times when experience gets in the way is when the reasons for decisions are no longer true, but you fail to revisit those decisions. My favorite example for this is the Jet engine:

Radial-flow compressors create far too much drag to create efficient engines. Axial flow compressors generate too little compression in each stage to create lightweight engines. Therefore jet engines are not practicable.

The above paragraph was true around 1930. Improvements in metallurgy, machining, and compressor design eventually made axial-flow compressors much lighter and more efficient. I can easily imagine a more experienced engineer telling a less experienced one something like "I looked into that and the math just doesn't work out."

One can't fault the more experience engineer because one doesn't have time to reexamine all of the underpinnings for all of the knowledge one has about the world on a regular basis. Though I do recall hearing a few people (including Feynman) who would keep a journal of problems they failed to solve. Any time they would learn a new technique or about a new technology they would flip through and see if there was anything in it that could be solved with the new information.

Another fun introduction to Lisp: http://landoflisp.com/

A great video from a great course ... not much has changed since it was recorded in 1986. (yikes - how I remember the hair-dos from those days)

This is from MIT OCW course Structure and Interpretation of Computer Programs [1]

[1] https://ocw.mit.edu/courses/electrical-engineering-and-compu...

Is there anyone out there who might be able to persuade Gerry Sussman to do a HN AMA? That would be great.

At 1:05:20 one can see a bug where (average 1.5 (/ 2 1.5)) evaluates to 1.3333 rather than 1.4166

Nitpicking aside, I would have loved to have this sorcerer among my CS professors!

Note that in the very next slide, the value 1.41667 is actually there, as the next tried value.

    (defun average (x y)
      (/ (+ x y) 2))

    (defun iterate (start)
      (loop repeat 3
            for val = start then (average val (/ 2 val))
            collect (list val (float val))))

    (iterate 4/3)
    ((4/3 1.3333334) (17/12 1.4166666) (577/408 1.4142157))
    
    (iterate 3/2)
    ((3/2 1.5) (17/12 1.4166666) (577/408 1.4142157))

Not sure if this is really a software bug, maybe a slide preparation bug.

How did they display the lecture slides on the in-classroom television in those days?

Most likely using overhead projectors.

It was quite some "fun" doing the printouts, and playing around to get them to display properly.

Good question and answer.

>It was quite some "fun" doing the printouts, and playing around to get them to display properly.

Ha ha, good one. I never had to do it (or had the privilege of doing it, depending on one's point of view), but saw it being done some (as a junior), for internal or client presentations at companies where I worked earlier. Another name for it was "transparencies", IIRC. Ugh. Fiddly stuff. Something like the difference between writing articles or books on paper (or typewriter) versus using a wordprocessor or text editor, with all the attendant slowness and rework in the former cases.

Note the statement about geometry teachers disliking inductive definitions at 01:03:02 https://www.youtube.com/watch?v=XYKRVNQ_MqE&t=1h3m5s

And then the same comment by another MIT professor 22 years later at 00:45:12 https://www.youtube.com/watch?v=SXR9CDof7qw&t=45m12s

Reply on news.ycombinator.com