Response to John Cook's "Myth of the Lisp Genius"
rondam.blogspot.com
rondam.blogspot.com
There are genius programmers who write Lisp,
and Lisp may suit them well. But these same folks
would also be able to accomplish amazing things in
other languages.
Extraordinary developers do extraordinary things, regardless of the tools used. Here's an example, using as reference http://en.wikipedia.org/wiki/Git_(software)#Early_history
The development of Git began on April 3, 2005. The project was announced on April 6, and became self-hosting as of April 7. The first merge of multiple branches was done on April 18. Torvalds achieved his performance goals; on April 29, the nascent Git was benchmarked recording patches to the Linux kernel tree at the rate of 6.7 per second.
And here's the kicker: Linus would probably be more productive in LISP, but would he bring more value to the world? Probably not, as churning out code is not Linus' strength -- he outcompetes everyone by superior design, by pragmatism and by being a good team-leader.
That's the type of developer I admire -- people that kick-ass in bringing value and making sacrifices is part of that. Sometimes you have the luxury of picking up a new language (Ruby) and invent a new web framework with it (Rails), sometimes you don't. Either way, code talks bullshit walks (so to speak).
Exactly the opposite actually. Not only do I not assume that, I specifically deny it.
I would suggest you are too much of an engineer to use an "inferior" tool like C++ or Java because you know there is something "better" otherwise I am sure you would achieve equally spectacular results in other languages.
Although I'm flattered, I have to ask: what possible basis could you have for making that assessment? My track record certainly doesn't support you.
> I would suggest you are too much of an engineer to use an "inferior" tool like C++ or Java because you know there is something "better"
A true engineer solves the problem under the constraints that he is given. I have a this tendency to try to change the constraints -- or the problem. Those can be advantageous personality traits for some tasks, but they make me a damn poor C++ coder.
"That still leaves open the possibility that I'm just freakin' brilliant, I could have (and would have) produced the same results in any old programming language. But there is actual data to refute that theory."
However, instead of invalidating his argument, he uses this to illustrate why Lisp is largely responsible for his success and why you don't have to be a genius to be attracted to Lisp or to benefit from using it.
...
> he outcompetes everyone by superior design, by pragmatism and by being a good team-leader.
On their face, these statements seem contradictory, don't they? How is it pragmatic to turn down a tool that yields better productivity?
Because individual productivity is not the most important thing when you have dozens or hundreds of people contributing to a project... or want to have dozens or hundreds of people contributing to your project. It may not be important at all, frankly.
Clever, compact code and extendable, maintainable code are often at odds, and lisps allow you to write very clever, very compact code in comparison to, say, Java and PHP. Or, probably, C, but I haven't written much C, so I won't opine on it. :)
Secondly, I dispute the notion that clever, compact code and extendable, maintainable code are at odds. Code becomes extensible by simple virtue of there being less of it. I do agree that clever, compact code is at odds with one thing: someone else looking smart. Do people write clever, compact code to look smart, or do you dismiss it to look smart? Because my experience is that it's almost always the latter. If I really did write the code in an unmaintainable way, they usually have more precise criticism than "It's too clever and compact."
"being a genius programmer", whatever that means, is not a one dimension thing, and people can excel at some things while sucking at others: programming is very vast.
I don't know much about LISP, and have not data to believe (or not believe) the OP claim, but he is not alone in claiming that. For example, P. Norvig has also described how he was surprised that people could be as productive in C++ as in LISP when he arrived at google (I think it was in coders at work ? He made a slightly differente claim concerning the unimportance of language on HN as well (http://news.ycombinator.com/item?id=1803815).
Lisp rolled forward my mind into new places of understanding how to program. It was like having lights flicked on all over the place. I've been able to conceptualize post-Lisp in ways I did not before.
Perhaps Haskell would have done similar things. I can't say. But Lisp definitely did, and I believe that given the project (and a sufficient lack of having to interface into weird libraries), I could do more in a given slot of time in Lisp than I could with other languages.
SML, OCaml, and/or Haskell will show you a new way of looking at types (and secondarily, expose you to unification on a level that is a little deeper than destructuring-bind). If your background is in perl and then some Lisp experimentation, I recommend checking these others out in order to fill in this other nice piece in the equation. Finally, once you have convinced yourself that type systems are the hotness, it will be good to expose yourself to Erlang, specifically with regard to error handling and the role of supervisors for process restarts.
Is it an attribute of Lisp that there exists a mythical lone-wolf that seeks to produce software so pure and elegant that no mere mortal can possibly understand it? Is it an attribute of Lisp that this lone-wolf, like a by-gone action hero, prefers to work alone?
Lisp, like any sufficiently useful language, simply gives you enough rope. It's good, sturdy, and well-tested rope. You could probably climb a mountain with it. Or hang yourself. Your pick. The rope has very little to do with it.
On those rare occasions when Lisp programmers manage to all row together in the same direction they can produce some truly spectacular results (Viaweb, ITA, RAX). But those seem to be the exceptions.
I wonder if the flexibility and power of lisp might work against co-operation in this way too.
I've witnessed confusion caused by operator-overloading in C++ (I heard that that's common); lisp is more customizable.
Yes. But that is not a problem unique to Lisp. It's possible to write unreadable code in any language. And in general Lisp's introspection capabilities and macroexpand make it easier to reverse-engineer things that in other languages.
Have you found that hard to understand, when it was heavily customized? I agree that tracing through complex code is taxing in any language; and it probably becomes much more important what concepts the author has chosen (and things like coupling/cohesion), than the language.
i.e. That the problem wasn't that the code was badly written, but it was because they had "customized the language so much". Extensive customization i how you're supposed to use lisp; it's an important source of its power. (The C++ example I mentioned was well-written.)
I can imagine it going either way (e.g. that lisp's customizations often aid clarity, by reducing code size); so I'm interested to hear someone's experience of how it actually goes in practice.
A good theoretical example is: imagine trying to understand what a CPS driven compiler is doing with no understanding of CPS. That would be pretty hard in any language, and it's actually better in Lisp because you can express things closer to the theory.
And yet good clean code is written in these languages, and they are successful for better or worse.
You mean like Java, .NET, Perl, and every other major project?
Maybe I can start a similar myth with Logo. It requires no proof. Simply one or two anecdotes, great hubris, and the ability to say with a straight face that the reason why everyone who uses it isn't productive is because we're geniuses. Anyone else in?
That's factually incorrect. There's plenty to show for it, you just have to scale to the size of humanity's computing concerns from four or five decades ago.
Moreover, in no field of creative human endeavor have popularity and worthiness been correlated. You think all those billboard top 10 acts came from Juliard?
Is this a troll?
It's not. Although I'll be the first to admit that it comes off as trollish. It's just I see these pronouncements about a language, which frankly, I don't think can be backed up. I like Lisp as a language. I really do. But to me that's not sufficient to annoint it a super language.
The evidence of its superiority is lacking. The one thing that languages generate is work product, programs/libraries/etc...
In contrast, look at something like VB. An oft-scorned language, but a language that you can walk into any Fortune 500 company and can probably find 10M LOC across the enterprise used for business critical purposes. And further, the interesting thing about VB is that you probably couldn't have replaced the language with any other language and had the same productivity.
Now of course this last assertion is hard to prove. But at least we have existence. At least we can say, "if there was a better solution than VB, no one used it". And it was rediscovered by every enterprise in the world.
Again, I'm not saying Lisp is a bad language. But to the best of my ability it seems to be about on par with other modern languages (and that in itself might be compliment enough for some, given its vintage). I see a couple of truly noteworthy Lisp projects, a lot of middling ones, and that's it. About what I'd expect for a language with a user base of its size.
Over the past 40 years there hasn't been any mind-blowing Lisp projects. And there's nothing wrong with that, I wouldn't necessarily expect it, except that its supposed to be a super language. Not only a super language, but a language practiced by the brightest amongst us. And by accounts of some evangelists, a toolchain that is magnitudes better than that available for other languages.
Yet as a result I typically get 3 projects noted to me when we talk about notable Lisp projects. Emacs, ViaWeb, and ITA. Three great projects. But if someone said that they were written in C and not Lisp, everyone would believe it. And when people go back and talk about important/influential software in history, Emacs is likely the only one to be mentioned. And of course there's been tons of other very important SW that has been written in languages other than Lisp, from Tex, to Mosaic, to Lotus 123, to gcc, to iOS, to Google, and so on.
The worse is better, Lisp super-loner, rationalization seems like a pity party. And what makes it sad is that it's simply not necessary. There's no shame in liking a language for characteristics you find appealing. But don't try to sell it as some super language. Because when you do people will say that the proof of the pudding is in the tasting. And the tasting are the programs. If you want to show that Lisp can do what others only dream of, show us the programs that make us drop our jaws. Show us the programs that make us rush to relearn Common Lisp. RoR didn't catch on because it had a sexy name. It caught on because the pudding was pretty tasty -- although a bit cloying for my taste.
Moreover, in no field of creative human endeavor have popularity and worthiness been correlated. You think all those billboard top 10 acts came from Juliard?
Are you arguing that for music that college serves little purpose? That we're entering some type of music education bubble. :-)
The post we are commenting on was by a person who was not making a claim, but relaying an experience. I will paraphrase three arguments: the parent post blog link was saying "look, I'm not an amazing programmer, but in my experience, armed with Common Lisp, I can approximate what I see ur-programmers do with other languages at Google." He in turn was replying to a person saying "I don't think Lisp makes you more productive." The "Lisp is not really a super language, it has nothing to show for itself, and I could make similar claims about logo." thing got interjected by you, I think.
"Moreover, in no field of creative human endeavor have popularity and worthiness been correlated. You think all those billboard top 10 acts came from Juliard?"
"Are you arguing that for music that college serves little purpose? That we're entering some type of music education bubble. :-)"
No, I think that's what you're arguing. I'm saying Lisp is worthy, and should not be judged by its popularity, because the worthiness is what reallly matters. You're arguing it has "nothing to show for itself". I'm sure with a Sun Java marketing budget and the right people we could make Lisp as popular as we wanted. We could have whitepapers and case studies to "show" as much as you like. I'm only concerned with popularity when people dismiss my arguments about worthiness the minute they assess the popularity. I have the same problem when I try to tell them about Sun Ra or the Residents.
So all that being said, to address your actual point (however disjoint from the OP it might be), nobody's claiming it's a "super language", just often better for the people who commit to using it despite it's unpopularity. I can back up my claims of betterness with arguments about worthiness, but not about popularity, but then I don't think the later kind of argument matters in the first place.
He was not only relaying an experience, as he cites the Lisp Curse. See:
"I do believe in the Lisp curse that the power of the language is in some respects self-undermining because it empowers the individual and so tends to attract people who don't work well in teams."
And I'm sure you've read the Lisp Curse.
His essay argues that he's really not a great programmer, but Lisp gets him to principal engineer! He further implies that Lisp really is something beyond other languages with his statement about the planner code (of code, he never considers that porting often fails even within the same language family for a variety of reasons).
I'm sure with a Sun Java marketing budget and the right pepole we could make Lisp as popular as we wanted.
How do you explain C, C++, Python, Ruby, Perl, Javascript, and PHP? None had very extensive marketing budgets.
So all that being said, to address your actual point (however disjoint from the OP it might be), nobody's claiming it's a "super language", just often better for the people who commit to using it despite it's unpopularity. I can back up my claims of betterness with arguments about worthiness, but not about popularity, but then I don't think the later kind of argument matters in the first place.
I don't know what your metric of worthiness is. Maybe its the existence of some esoteric feature. You say I talk about popularity, but I only do so to ask for an example of the "worthiness" you speak of. As I said before the proof of the pudding is in the tasting. If you tell me you have some great pudding, but it tastes like cow dung, I won't be impressed regardless of how many fine ingredients you use and advanced cooking methods employed. It still tastes like cow dung. And I'm not saying that Lisp is cow dung, but I am saying that fancy ingredients and world class ovens don't impress me if you're making the same McDonald's sandwich as everyone else.
And when you start creating great pudding, you won't need the marketing budget.
I think a lot of these have larger marketing budgets than you imagine. I can't fathom the amount of money Microsoft spends promoting its C++ language tools, for example.
Regardless of whether languages other than Java become popular or not because of marketing, I still maintain that popularity and worthiness are not correlated. The majority of people may not eat cow dung, but they certainly don't eat really great pudding. They eat the same so-so pudding everyone else does, and most of them don't realize or don't care that it could be better. You will have nothing to "show" for coming up with a better pudding recipe unless you spend money on advertising, manufacturing, distribution, etc..
As for an example of worthiness, Lisp's advantages have been detailed one metric kerjillion times elsewhere. Macros, conditions, and the MOP are the usual suspects in the case of Common Lisp.
I think he's asking, if Lisp is so awesome, why doesn't someone ever do something awesome with it? It seems like a fair question.
But still everything was not rosy. * When we started, documentation was a huge problem. We would never have got approved except for Edi, who's efforts were an inspiration and a basis of our stuff. * Library support was sparse. Now it is a lot better. But the community is small. Not that they are all pricks, but they are stretched too thin. If I have seen anything, if they use lisp for work, most of them have to fight daily battles against the anti-lisp crowd which will not look rationally at any good work you have done. * Library management - now much better because of quicklisp (thanks Zach) * For most people lack of a usual ORM and a MVC framework was unthinkable. CLSQL has been a pain point.
Programming in the Lisp environment, with the REPL, is a pleasure. I have had to do php after than and O-M-G - its horrible. All I will say to other fellow Lispers - The naysayers are always present. Trust your instinct. You will enjoy life better if you are able to love what you do.
(Cleartrip does not run on CL now. Java, I think. Because of reasons other than technical IMHO.)
So therefore Perl > Lisp > everything? (I'm kidding for the humour impaired)
What he gained from using Lisp was not worrying about memory management, and built-in data structures that made life easy. These days you can get that with a lot of different languages (and I would argue that Java is not one of those languages), languages that also perform as well as Lisp.
So what's my conclusion? That I think there are both - genius programmers who can code well and quickly in just about any language, and also languages that let you build things a lot faster than traditional languages.
It is not just about memory management and data structures. It is about functions as first class objects, no side effects, lazy evaluation, closure, monads. The fact that you can basically write a compiler in an hour as a first year student (that was exercise 3 in my course).
I'm not a programmer, I'm a DBA. The code I write is mostly short and ugly scripts in Perl and Bash. I can do very little Java and almost no other language. The one and only time I used Lisp was in my first programming course in the university. Incidentally, it was also the one and only time I enjoyed programming.
I learned FP using Haskell, which is a pure functional language.
But I still stand by what I said. I code faster in Perl than I do in Haskell due to the availability of libraries. The "pure FP" features may make some things theoretically cleaner, but they don't honestly make things that much easier. Having access to FP features (map, grep, reduce, closures in particular) is significant, but access to built in data structures (hashes in particular) make life so much easier than C or C++ or Java.
To say that the original article was "about" Lisp, or the Lisp Genius, would be akin to my saying that there are fleas "about" my dog.
BTW: I wonder if Rondam(?) knows this guy: http://www.flownet.com/gat/jpl-lisp.html
an attempt was made to port part of that code (the
planner) from Lisp to C++. After a year that effort
had to be abandoned because the planner team just
couldn't make it work.
I'm doomed.
stop linking to John Cook.
Every John Cook blog entry I have ever read turns out, upon inspection, to be an almost-content free restatement of an accepted consensus -- motherhood statements at best.
Whereas Jeff Atwood at least gives credit to the books he reads and summarises, Cook merely regurgitates unsourced factoids and leavens with a few unimpressive anecdotes.