What Made Lisp Different (2002)
paulgraham.com
paulgraham.com
The first was a Common Lisp web app for resource management, that was recently sold. Not a giant payout, but over it's entire life, starting in 2000, it paid peoples mortgages and put kids thru schools.
The second was a clojure based malware analysis engine, which we sold to Cisco about 3.5 years ago. That was a more conventional startup exit. We have scaled that into a global product. Additionally, we are developing even more clojure products within Cisco, and hiring...
I know from hiring clojure devs for the last 6 years that there are alot of companies, big and small, using it.
Could you expand a bit on this? I'm not entirely sure what you mean, but sense some good advice.
Lisp, due to its easy metaprogramming, gives abstraction enthusiasts rather more rope to hang themselves than languages which make abstraction inconvenient (e.g. Go, which has a simple type system, or Java, which is very verbose). So, if one isn't careful, it's possible to write a very clean, very nice Lisp DSL which is totally incomprehensible to anybody but the author, and thus hell to maintain over time.
Relevant quote, from https://www.joelonsoftware.com/2000/07/22/microsoft-goes-bon... –
> When great thinkers think about problems, they start to see patterns. They look at the problem of people sending each other word-processor files, and then they look at the problem of people sending each other spreadsheets, and they realize that there's a general pattern: sending files. That's one level of abstraction already. Then they go up one more level: people send files, but web browsers also "send" requests for web pages. Those are both sending operations, so our clever thinker invents a new, higher, broader abstraction called messaging, but now it's getting really vague and nobody really knows what they're talking about any more.
> And if you go too far up, abstraction-wise, you run out of oxygen. Sometimes smart thinkers just don't know when to stop, and they create these absurd, all-encompassing, high-level pictures of the universe that are all good and fine, but don't actually mean anything at all.
More here: http://wiki.c2.com/?TooMuchAbstraction
You can't really call it a "very nice" DSL if it is incomprehensible and unmaintainable :)
What you have described is one issue, but it really doesn't have a lot to do with lisp DSLs. You can make that mistake in any language.
A problem that is more specific to lisps is that since it really is possible to write beautiful programs, there is a temptation to spend a lot of time designing DSLs or otherwise refactoring your program to make it very clear, concise and maintainable - but without actually making it more useful. You may spend hours (or weeks) improving your code without actually improving functionality at all.
This is less of a temptation in Java, for example, because it's such a rubbish language to program in in the first place there's very little temptation to spend time refining your code. You just accept that your program is never going to feel like a work of art no matter what you do, and get on with actually producing a useful solution.
This is a myth. DSLs in Lisp are almost always done in Lisp syntax, no surprises. The idea of having a higher abstraction is to make the problem domain map more clearly to the code.
>totally incomprehensible to anybody but the author
Now, what people who like to repeat this myth forget to say, is that when using an OOP language, say, Java, and using another's person code, one also needs to learn the classes that this person has created, the methods and how they interact. And this can be "totally incomprehensible to anybody but the author".
>Relevant quote
No, not relevant. Joel Spolsky is talking about device abstractions.
React isn't a DSL; React includes a DSL for html generation.
And by the way, we have had such a thing in Lisp since the existence of HTML. Tons of libraries that allow you to inline HTML into your code.
You are assuming that a DSL is done by using macros; which is incorrect; in Lisp you can easily create your own DSL using plain functions as well.
> The idea of having a higher abstraction is to make the problem domain map more clearly to the code.
Certainly. And, as per the Spolsky quote, too much abstraction can actually miss the expressivity sweet spot and start to obscure the problem domain.
> No, not relevant.
It is if you, ah, abstract away the specific context of his story.
It always is.
Code isn't magic, so you have to write whatever logic you would write in one language in another, for any application you would write.
The only exception if your language provides abstractions on which you can build on. For example in Perl, you can do a lot of file operations and regex work without libraries. In Java you might need a library, sometimes a combination of libraries, whose documentation one might have to read.
Same goes with lisp too. The only difference being, every once in a while one might discover a way to do large patterns of code through simple ways using macros.
I also use it to handle my contracting finances. Let's grep that software for "macro":
$ grep 'macro' *.tl
money.tl:(defmacro define-relational (name binary)
money.tl:(defmacro define-arith-predicate (name usr-package-fun)
money.tl:(defmacro define-unary (name usr-package-fun)
time.tl:(defmacro def-date-var (name fallback-val . date- range-val-triplets)
time.tl: (defsymacro ,name (range-lookup *date* ,fb ,rvt))))
The bulk of the system is OOP code: accounts, transactions, expenses, invoices, deltas, ... things with boring stuff like slots and methods. (Of course, I wrote that object system with a fair bit of contribution from macros; but that's in the language, not in the project).
The DSL for recording transactions consists of plain old functions with transparent contents: just instantiate certain objects, do certain calculations and insert into the ledger.
The date-specific variable thing is cool; it lets us define a variable, such as an income tax deduction rate, which has different values based on what date it is. Not the current system date, but the context date for the creation of a transaction: the date to which it is being accrued. A macro helps define that with a clear syntax, and the symbol macro makes it look like an ordinary variable.
I think the advice given in this thread's root is good: Stay close to the problem domain. Really close. If you wonder whether you have gone too far, you probably have.
[1] You of course can gather HTTP-headers via :list method combination. Seriously, you can. And then comes reality in the form of some closed source middleware that expects headers in a specific order and it even refuses to discuss elegance or adhering to protocol specs with you.
Rigetti hosted one of the Lisp meetups to talk about their use of Lisp and it was recorded in a talk "Lisp at the Frontier of Computation" [0].
[0] https://www.youtube.com/watch?v=f9vRcSAneiw
(Disclaimer: I'm the speaker in the video.)
I have no relationship with Lisp, YC, or Rigetti.
At the same time, one of the reasons I don't like it too much for daily work, is how it encourages cleverness. This sounds silly, I know. But sometimes when there's opportunities for cleverness -- such as with C++ (meta) template programming -- I become too tempted to make my own code more clever instead of, you know, making it do whatever it's supposed to do. That makes me appreciate the relative dumbness of say, Go and Python (not that you can't be clever with Python, but it's not encouraged).
I fully agree: nothing is as dumb as being too smart.
From a linguistic PoV LISP stole my heart as a lad and never really let go... I think it's much more consistent and reasoned than most programming languages, and it's wildly powerful. I can only imagine what 5+ years hacking a domain on a proper LISP machine would bring...
That said, I can't bring myself to use it in Enterprise projects because of the 'LISP curse'. While LISP got computing right, I think the type systems of the ML languages got industry right. They strike a nice balance between linguistic power, wild cleverness, and a brutal mandatory type system that means I rarely have to understand cleverness. As long as a function signature says it turns a 'Foo' into a 'Bar', I've got what I need.
You have to create a DSL for your problem, no matter what programming language you code in. Its just that most people never program in anything beyond their area of exposure, and their DSL which they creates just appears very normal to them. But not to others.
I kinda liked it for a systems language. It has conveniences trickled down from higher level languages[1] that are missing from C & C++ and which make them such a chore to use
[1] for example: garbage collection and being able to print an object without have to import vast libraries
There are garbage collection libs for C++ and they are used by C++ programmers whenever necessary.
ML languages strike a pretty nice balance here, IMO. Your DSL tend to be normal code with a funny operator or two, structured into happy little lists.
I claim a lot of problems like this are solved perfectly well by team-agreed policy. For the same reason you don’t see people writing C functions taking void* everywhere for every problem, you don’t see professional Lisp codebase chock-full of terrible, homegrown DSLs.
You will see bad code in Lisp, C, Go, Java, whatever if it goes without review or peer agreement. With that said, since coding in Go is a bit more menial, there’s less policy to make around it.
I avoid cleverness, including when writing Lisp. I rarely use macros, for example. Even if no one else reads my code, I'll still have to understand how it works in future.
But in enterprise OO software (-> Java) you can get similar effects once there are enough layers of factories, special purpose DSLs, greenspunning, etc.
'can' is the wrong word there. You 'always' get an effect worse than lisp.
It once took me hours and severals hundreds of classes to sift through a massive java code base to understand that all the programmer wanted to do was post a XML to a endpoint.
This is more of a software engineering problem and not a lisp problem.
Every so often, someone writes an essay explaining why, if Lisp is so good, it isn't used much. There are at least three others.
Unfortunately some of the Lisp cool features are hard to implement without its simple syntax. Maybe the lesson is that we should find a f way to have syntax-independent programming languages! Just as we expect any successful language to have at least a couple implementations before being taken seriously, maybe we should expect a language to have at least a couple different alternative syntaxes and instant perfect translation between them (with comments and code style preserved), so I can use syntaxA and work on same codebase with my colleague using syntaxB. Than we'll be sure any metaprogramming or code intelligence tools don't suck because they'll be damn forced to not use the program text but the AST (that will have to be standardized) as they should instead...
Lisps would not even seem so cool anymore if we could get our s together and build languages with (a) standardized AST representations and (b) multiple syntaxes... we've like already imported most lisp features into modern languages anyway, and standardized ASTs would make macros both trivial and manageable...
life←{↑1 ⍵∨.∧3 4=+/,¯1 0 1∘.⊖¯1 0 1∘.⌽⊂⍵}
To put it the other way around, it reveals something about how choice of programming language affects not only how we write a program, but what we write, and even how we think about the problem.
Interviewer: "Wouldn't it have been possible or simpler to add this using some kind of library or framework to an existing language? In other words, why does this kind of stuff need language support as opposed to just a library?"
Rich: "In fact it is a library. It's a library written in Java that you can use from Java. The whole thing about a language is, a language is about what does it make idiomatic and easy. So for instance, you can use the same precisely the same reference types, and the STM, and the data structures of Clojure, all from Java... The lack of idioms and language support means using exactly the same constructs, the same underlining code, from Java, is extremely painful compared to Clojure where it is the natural idiom."
I've always loved that definition of a language.
[0]: http://www.se-radio.net/2010/03/episode-158-rich-hickey-on-c...
Consider assembly or basic, which use a lot of goto-statements. It can get very ugly trying to fit some arbitrary goto's into the more structured loops and if-statements we're accustomed to.
The Sapir-Whorf hypothesis is certainly real in programming languages[1].
There was a lot of work around equivalance of control flow between goto/conditional branch instructions/unconditional branch instructions and "while programs" in the 70's that showed there's actually a limited variety of control flow patterns, and while programs can be made equivalent. I think the result of this is the nowadays presence of break/continue loop constructs in most languages. Maybe this is a folk theorem, I don't recall really.
Even if one of the syntaxes is not text based, maybe even a "syntax that involves arranging blocks in 3D space with a VR headset" or whatever, people could still agree on sensible defaults.
Only issue I'd see might be with open source projects, where you could add a guideline like "please reformat your code so that it looks good in syntax X according to styleguide X-42 or your pull request will not be accepted". Yeah, the conversations might become less democratic, with some "style dictator" needing to push "the right rules" down right every contributors throat to prevent endless bikeshedding, but it could work.
If we'd give of some of this crap "style democracy" and have a few more authoritarian rules, we could instead enjoy multiple syntaxes easily. The ol' "more freedom under tyranny" paradox that people always seem to (like to) forget it actually works, and could work well to code too.
Microsoft tried something very similar with C# vs VB.net (in the early days of .net there were even more experiments like J#). There exist tools to convert quite seamlessly between C# and VB.net code.
The result: except for some geeks people have decided that they prefer the C# syntax and it seems to me that Microsoft thus slowly tries to fade out VB.net.
So it rather seems to me that people don't like it if there are multiple coexisting syntaxes, because even if there are good converters available, it is still an inconvenience that wastes at least some productivity by mere existence of the multiple syntaxes.
I suspect this would work best for an image-based smalltalk-like system where the “source of truth” would be the version of the code persisted in the image and then the textual representation would be generated from that on demand.
Not really, VB.NET enjoys much more MS love than F#.
Just check the tooling and documentation at MSDN, or the set of languages supported for UWP development.
> Not really, VB.NET enjoys much more MS love than F#.
This is surely true, but in my opinion F# is more different from C#/VB.net than "just being for most parts a different code representation". So I would consider F# as an independent programming language - which indeed seems to be fallen out of favor by MS.
It's one thing to implement all kinds of features and another thing to integrate them well. You can bolt wings onto an existing ship, but it won't fly well.
You can't build an airplane from bricks.
-- Steve McConnell, Code CompleteThis makes it sound as if there are no modern lisps.
What's the big deal of using a lispy language? Do other languages really need to desperately try to reimplement features that come more naturally to lispy languages? Are parentheses (only the round ones) really that much of a deterrent to motivate excessive work like that?
I'm very happy with Guile and Racket.
It seems to me that very few companies use Lisp-like languages used in production. Even Haskell seems more common.
The big deal with using Lispy languages is that companies don't want to use them -- it's a lot easier to hire developers who know other languages. So, I'm stuck with Java whether I like it or not, and I appreciate any functional programming features that can be packed in to new Java releases, even if they end up being a bit clunky.
The reason why adoption concentrates on a few languages is mostly network effects rather than objective features of the language itself.
Maybe it's time for us to think about how to increase the adoption of languages rather than trying to bolt every feature onto java just because there is a lot of java code and devs. We're going to be stuck in a never-ending mess of complication and legacy.
I don't know of any way to translate between the two seamlessly; the translation itself would be pretty easy but preserving whitespace would be tricky.
I'm kind of put off after having tried Haskell and seeing now OCaml as an "inferior Haskell", and after looking at F# and seeing it as a "better integrated OCaml". Also folks in ML and data science, the area I care more about now, seem to like F# better, not much mention of OCaml here.
My experience was just that H felt so elegant and conceptually simple... up until a point (like when you start needing to use lenses or think of monad cobinators). Even the syntax was so nice and readable (love the special monads syntax, the `when` statement etc.).
OC's designers seemed to not be very fond of the concept of simplicity, simplification, reduction, elegance, improving by removing stuff etc. Only on correctness/soundness. Sounds very... French :) Not a bad thing (I live in France now and like it). Just I don't like it in code. I'm more of a "let's symplify shit more and more until maybe the problem goes away, completely, by itself, instead of actually starting to implement a solution, from the get go" person.
Oh, and yeah... I don't think I'd ever write a production system in a lazy language, I agree strictness is a feature. (Maybe I'm biased, but I just like to be able to reason exactly about when some code runs, whether it has already ran by the time a breakpoint is reached and all that old school stuff.)
This is somebody else's point I read, but there's also a cognitive dissonance between 1. Haskell is pure, so it's very easy to reason about anything and 2. Imperative programming is just as easy in Haskell as any other language if you use the do notation.
Could you give an example of OCaml's designers removing too much stuff? Are you talking about the lackluster stdlib here haha?
As for laziness, I agree. I like lazy semantics (I'd argue they might almost be strictly superior than strict semantics), but there's too many things it sacrifices. Being able to reason about resource/time composability, having stack traces, having a debugger, etc. is amazing.
I mean, having to write + or +. or +/ instead of having something like a Typeclass for nice polymorphic operators? Feels like an annoying straightjacket and syntactic noise. And I don't understand the tradeoff: since I see types a lot like "compiler checked documentation that can also work as basic tests", I see no value in type inference for functions - I'd rather have only local type inference for variables, specify manually all the types of all the functions (this helps you think better anyway) and instead be free to binge on polymorphic operators and functions/methods.
Haskell almost seemed to deliver that with its typeclasses. But then its laziness and monads and weird handling of records make it too weird for practical use for me.
Is there an "OCaml with typeclasses or other form of polymorphism"? :)
As far as the tradeoff, my view on types is a bit different than yours, I think. First, a quick note:
> specify manually all the types of all the functions This is usually encouraged in OCaml, in the form of a .mli file.
I think types can be a lot more than "compiler checked documentation that can also work as basic tests". They allow you to encode invariants that allow the compiler to check more than basic stuff. I thought this video had a good example. https://www.youtube.com/watch?v=-J8YyfrSwTk&t=20m10s
I think one of the problems with getting new people into OCaml is that you feel the pain of not having typeclasses long before you understand the power of functors.
All the mechanics of how values are created, interact, and are torn down are described by types, whether the types are dynamic or static. It's the type systems that are the limit to language interoperability, not syntax.
I could go further: it is object systems that are the biggest problem. Program with inert data structures and pure functions of input to output, and interoperability is much more easily achieved. But if you expect to create an object whose behaviour is defined according to language A, and poke it into a function that interacts with it according to the idioms of language B, effectively smuggling A's semantics in via the polymorphic type, somebody is going to be surprised.
And semantics are what people care about, too. Syntax does have its bikeshedders, but semantics are what make people move or stay.
There are at least two big exceptions to this rule:
- 1. Lisps - some people (some of them really smart btw), just can't stand lispy syntax (I'd say that it's most likely because they got the math/physics notations so deeply engrained in their minds that they can't tolerate breaking apart from "thinking in it")
- 2. operator overloading heavy syntaxes - when writing any kind of sciency code, you'll always have a camp of people that will want to use 50 2-letter operators instead of functions everywhere, and camp of people that will just want "zero tolerance for operator overloading". they'd both have their reasons, and there will be no way to "make peace" -- in a multi syntax setting you'd just have one syntax that will show the `plus` function/method used and another one that would render it as the `+` operator
I'm not sure what a standardized AST would really be. A standardized concrete syntax tree is doable, but each language is going to have its own statements and expressions (equivalent to special forms in Lisp). This is similar to how we can use JSON objects to represent all sorts of different types as key-value sequences with different sets of keys.
For an evolving language, the AST needs to change from one version to the next. When you add a new kind of statement or expression, the tree-walkers in the tools usually need to adapt. The tools aren't stable unless the language is stable.
- Comma-separated lists: a, b, (c, d), e
- Square-bracket lists: [a, b, c]
- Blocks (curly brackets and separated by newlines or semicolons): { a b; c d }
- Dotted lists, where sometimes the dots can be omitted: foo.bar.baz
In combination, you can write something like:
for x in [1, 2, 3] {
foo.bar(x + 1, x * 2)
}
I've implemented this, but I'm not satisfied with it; the corner cases are tricky to understand.
Haskell's type system is more expressive, so information is lost in the translation to C (that the haskell int is non-null), thus becoming maybe int when you translate back to haskell
I can't work on a codebase with a colleague, me writing/reading the code in Scala, he/she writing/reading the same code in Java or Kotlin. I can't code a project in Clojure, then hand it over to a team of programmers that see it and work on it as Java code.
JVM languages are too different from one another (and the "common language" underneath, whatever is it called, it's waay too low level to qualify, practically no one writes/reads it). Different syntaxes would mean need to share a different semantic common to all to have seamless translation. (And yeah, apart from some academic experiments I think we're far from abstract syntax independent semantics in any production used programming language.
To actually be able to treat the multiple JVM languages as syntaxes of the same language with a "seamless" experience you'd need code analysis tools of almost-superhuman intelligence. By that point we'd be out of job anyway ;)
I think the main reason that this wouldn’t work is that compilation isn’t perfectly reversible. Information is lost.
I think you will always have this problem with translation and it’s a analogous to the idea of two people working on a novel, one in French, the other in English, with neither knowing the other language, expecting the word processor to come up with some lower representation that it can use to translate flawlessly back and forth.
http://www.paulgraham.com/avg.html
> The more of an IT flavor the job descriptions had, the less dangerous the company was. The safest kind were the ones that wanted Oracle experience. You never had to worry about those. You were also safe if they said they wanted C++ or Java developers. If they wanted Perl or Python programmers, that would be a bit frightening-- that's starting to sound like a company where the technical side, at least, is run by real hackers. If I had ever seen a job posting looking for Lisp hackers, I would have been really worried.
17 years later and I still don’t see all those Lisp startups :)
I’d argue that 8) will never happen because the gap between human parsing and computer parsing is too big. Except for a minority of human compilers that enjoy Lisp, most people don’t, so this notation will always be a niche for mainstream programming.
Just survivorship bias from PG side (lots of Lisp based companies just gone nowhere) and gross extrapolation of the sole example his one Lisp company as some universal truth (and even for his company, it was more the idea, timing, skills, execution and luck, than Lisp of course).
The youngest of the companies you list was founded in 2007. The first stable Clojure release was in 2009.
When one of those dialects is completely source-incompatible with every other dialect (the rest of them share non-trivial programs freely), and doesn't share the same core features as every other dialect (like list structure), then why is it still a dialect and not a different language? Calling all the other Lisps that share core features and run the same code different versions of one language makes sense to me; I don't see the point in applying it to any language that borrowed a couple ideas.
If I have a large codebase written in (any) Lisp, will porting it to Clojure be significantly easier than porting it to Javascript or Ruby or any other modernish language? Not really; it's a complete rewrite either way. That sounds to me like it might just be a different programming language.
Heh. Most Scheme dialects don't even have source compatibility despite implementing the same standards.
Common Lisp lets you write portable (between different CL implementations) applications, but it's easy to end up doing something that is implementation-dependent if you're not careful.
I wouldn't say that. I've done a fair amount of porting code between CL implementations, and it's usually pretty straightforward. Implementation dependencies tend to be found only in code doing OS-y things like threads, external processes, network sockets, etc. This code is usually not difficult to find, and I would think that the authors would have expected it to be implementation-dependent. Computational code usually ports with zero effort, even when the machine word size changes.
"C combines the power and performance of assembly language with the flexibility and ease-of-use of assembly language."
Most languages are a evolutionary dead end. They won't even get this far.
I also wrote other comments on other sites using software written in PHP, what does that have anything to do with anything mentioned here? I guess it proves that you can use Lisp to write forum software. Still, not really related to my main question.
Slightly off-topic, I should mention that I've read most of pg's articles several times over the years. I know about Arc, Viaweb, etc. I'm asking about this topic after having read all those articles.
> The youngest of the companies you list was founded in 2007. The first stable Clojure release was in 2009.
Ok, then some other big company founded since 2009. It's been already about 9 years, there should be some around. I can wait for examples. Those companies can use any dialect of Lisp invented since the 50's :)
Clojure is not based on CONSes but you have CAR and CDR they are first and rest. Processors are no more based on accumulator and decrement registers... :D
Cons exists but it works only on lists as CONSes doen't exists in Clojure, only lists.
Empty list != nil.
Clojure is based on immutability like Scheme, CL is often used in a mutable way (see the shameful f-set).
{}, [], #{} are syntactic sugars and have already been implemented in CL.
Clojure is an hosted language, so Clojure, ClojureScript, ClojureCLR can be somewhat different, but the core language is the same (same base Clojure library).
CAR and CDR are operations on conses, so you can't have them without conses; that's like having sqrt without numbers. Lisp has FIRST and REST for operating on lists, and so does Clojure. Lisp also has CAR and CDR for operating on conses, but as Clojure doesn't have conses, it doesn't have CAR and CDR, either.
> Processors are no more based on accumulator and decrement registers
CAR and CDR were named for 'contents of address part of register' and 'contents of decrement part of register', not anything about accumulator or decrement registers. I'm not sure where you heard that processors don't have accumulators anymore, but that's definitely not true. The reason Clojure doesn't have CAR and CDR doesn't have anything to do with obsolete processors, though.
> Clojure is based on immutability like Scheme
Scheme is not based on immutability.
> CL is often used in a mutable way (see the shameful f-set).
Do you mean SETF? FSet[0] is an immutable data structure library for Common Lisp. A lot of Schemes have SETF, too, by the way, there's even an SRFI[1]. But SETF is just so that you can have one generic assignment operator; standard Scheme has plenty of commonly-used, destructive assignment operators like set-car!, set-cdr!, vector-set!, etc.
0: https://common-lisp.net/project/fset/Site/FSet-Tutorial.html
Scheme isn't based on immutability.
R7RS:
> Literal constants, the strings returned by symbol->string, and possibly the environment returned by scheme-report-environment are immutable ob jects. All objects created by the other procedures listed in this report are mutable.
> {}, [], ...
These characters are reserved for the user in CL and have been used for a bunch of different things.
You can't have atom without conses (well, it would just be (lambda (x) t)); which Clojure function are you thinking of?
Almost. It deviates a lot from all the other Lisps, in particular the two main branches: Common Lisp and Scheme.
However it is still a Lisp family language, and thus with its advantages.
Objective-C is only a mainstream language now because Apple is the most successful company on Earth. No one besides Apple was seriously using it at the time, and they used it to develop the most popular product in the history of computing. And of course, then bet their development future on an entirely new language, Swift, which of course no one else was using, because Apple invented it.
True, it's crazy Facebook had so much success with PHP. But they are also one of the biggest corporate users of Haskell, I think.
True, on the surface Amazon and Google are on the stodgy side with their "supported" languages. But I believe Google internally is rife with custom, proprietary DSLs to drive a lot of their infrastructure? And they invented Go. Which is in many ways the opposite of Lisp in terms of design. But the larger point PG was making was that the dangerous technology companies evaluate technology decisions on the real potential productivity gains, and not on the number of job postings for that technology.
EDIT: Saw Erlang mentioned elsewhere in this thread, which reminded me of WhatsApp, which led to 50 engineers supporting 900 million users and a $19 billion payday from Facebook! That's the perfect example of the kind of company PG would have been worried about.
[1] If you call iRobot is a venture company. It is actually 30 yrs old.
- Peter Norvig
After a full year writing Python professionally, and then Common Lisp, i can confidently say this is not true.
- true, effortless metaprogramming
- speed close to C by use of type declarations and fixed arrays; this means up to 100x faster than CPython. No, PyPy, Jython and Cython don't get quite there.
- no Global Interpreter Lock!
- the condition-restarts error handling system, which is deluxe error handling and recovery from errors.
- a very flexible type system, plus very strong typing.
- an extremely powerful object oriented system (CLOS) -- light years ahead of most OOP systems including Python. This means, having multimethods/multiple dispatch, method combinations, around/after/before methods, multiple inheritance, and a MOP.
- true lambdas, not Python's one-line joke lambdas.
- true interactive development, able to compile functions on the fly and thus change them while the code is running. Able to redefine classes while the code is running.
- able to call Java libs and C libs at the same time, with ABCL
- will run on many platforms and in the JVM without modifications to the code
- separate namespaces for functions, vars, keywords and everything
- tail call optimizations
- something like Quicklisp (pip isn't as good)
- built-in rational numbers, complex numbers, using the same operators for regular numbers
- the LOOP macro, which I find better than Python's list comprehensions.
In what way are Python's lambdas restricted? Do they literally have to fit on one line?
Do you have to use a separate version of + to add complex numbers in Python?
Do you prefer loop because it's more concise or because you can say things you can't say in Python? Can you give me a couple examples of things that work better with loop?
(As you can tell from my questions, I don't know much about Python, but as a Lisp hacker I'm curious about other Lisp hackers' view of it.)
You can define inner functions and so on, and most of the times that I want to use lambdas it's just because I've been stuck in a language without *-comprehensions for too long.
+ works fine on complex numbers.
I'm not super familiar with the loop macro, but it lets you do both things similar to a python comprehension and to a python for loop. I don't think it does anything you can't do in python.
(To be clear, decorators aren't macros, "decorator" is a macro)
I don't see where in the article he is claiming that you would see many Lisp startups in the future.
>But I don't expect to convince anyone (over 25) to go out and learn Lisp. The purpose of this article is not to change anyone's mind, but to reassure people already interested in using Lisp-- people who know that Lisp is a powerful language, but worry because it isn't widely used. In a competitive situation, that's an advantage. Lisp's power is multiplied by the fact that your competitors don't get it.
I’d argue that today libraries and super used code (more bugs already found and fixed, more answers on Google) > powerful language.
I'd argue that libraries that used to have bugs found and fixed tend to have many more of them to catch you by surprise later. They also tend to have less stable interfaces.
> more answers on Google
Also, the libraries complex enough to need many questions on Stack Overflow are normally the same as the ones above.
http://www.flownet.com/ron/lisp/djbec.lisp
and take a look at the functions xpt-add and xpt-double. The details don't really matter. What matters is that what is going on here is a whole bunch of modular arithmetic. The details are described here:
http://www.hyperelliptic.org/EFD/g1p/auto-twisted-extended-1...
Notice how the Lisp code is virtually cut-and-pasted from that page, despite the fact that it's not Lispy syntax, and the mathematical operators are NOT traditional addition, multiplication, etc. These are operations on modular integers and elliptic curve points. And yet the code looks just like the source material from which it was derived.
To make all this magic happen required less than 500 LOC.
+1
A different question would be - what would a Smalltalk like system look like 'in the large'? (There was the Croquet project that was a live distributed environment so there has been some work done here already.) It seems the hallmark of Smalltalk was late binding so perhaps the core model can be extrapolated to a loosely coupled distributed system?
I do wonder if the work at Yale on Linda[1] & Tuple Spaces[2][3] would have been more popular when it was introduced in Java[4] then people would have reasoned a bit differently on big systems. Going the CORBA[5] route always seemed to generate really problematic systems.
The again, I really thought it would be cool to have people program in Occam[6] just to force some thinking about organization of programs.
1) https://en.wikipedia.org/wiki/Linda_(coordination_language)
2) https://en.wikipedia.org/wiki/Tuple_space
3) https://books.google.com/books/about/Mirror_Worlds.html?id=j...
4) https://www.javaworld.com/article/2076849/core-java/jini-tec...
> work at Yale on Linda[1] & Tuple Spaces
Ah, I see what you mean by loosely coupled. Yes, Linda is neat and if we think about objects/messaging with nested spaces, then each space can have its own form of messaging (could use tuple spaces, for instance).
Also, I really like the cluster of VMs idea because you can then implement various kinds of cluster wide virtualizations on it (Croquet did this with Squeak VMs). I do think that cluster wide virtualization is the future of large programmable systems. We've built up some abstractions that work 'in the small' - we don't have to write assembly or do register allocation, but most 'languages' still limit you to think about what goes in within one Unix process, a very small part of the system. Whole system programming abstraction aren't really there yet.
Yeah, but I think its more a problem with the original everything is an object concept. I've thought about it a lot, and keep coming back to Animal Farm[1] paraphrased as "All objects are equal but some objects are more equal than others." We create objects that really do things and value objects, but it takes time to examine a program and figure out the structure of the objects. Telling the doers (often name Agent or Manager) is a pain. Its not implicit in the programming language.
> Yes, Linda is neat and if we think about objects/messaging with nested spaces, then each space can have its own form of messaging (could use tuple spaces, for instance).
My thoughts on it are that you could really use the tuple spaces as the VM boundary (communications between VMs is the tuple space). Plus, the tuple space provides an interesting way to decouple logic and actually replace agents in a running program. Freeze the queries to a tuple space, replace the agent waiting on the tuple space, then unfreeze the tuple space. I think building a language and patterns to not only allow maintenance but also facilitate program upgrades would be very helpful. Not to mention testing of agents outside the whole of the program. A test bench where you can load an agent, connect it to a tuple space, and then run a ton of data through it to perform units tests would be very interesting.
I agree with this too. I do think there is another layer of organization needed, which can be inspected to see the high level structure of the system - which objects form which part of the interconnected graph and which objects are passed around as messages, etc. I don't think everything-is-an-object needs to be changed though. An old attempt at layering another organization on top of objects is in the PIE papers: http://esug.org/data/HistoricalDocuments/PIE/PIE%20four%20re...
> My thoughts on it are that you could really use the tuple spaces as the VM boundary (communications between VMs is the tuple space).
Oh I see, interesting. The 'freeze queries, update, resume' ideas sounds very useful, as similar patterns are often re-implemented many times over by various distributed databases. Might as well make it a standard feature of the system. One question is why only apply this to VM boundaries? Could this be applied to smaller scale (e.g. between smaller object clusters within one VM?). Applying this with finer granularity might let us update a single object or method only.
The 'freeze' idea seems to fall into the managed time concept. Some interesting work in this area is Virtual Time by David Jefferson - http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.134... - and also NAMOS by David Reed.
EDIT: like a replacement for Jupyter, but more capable.
1) https://en.wikipedia.org/wiki/Telescript_(programming_langua...
2) http://robotics.stanford.edu/~shoham/www%20papers/AgentOrien... from http://robotics.stanford.edu/~shoham/
Does it? Maybe it's the way I think, but I have observed that I could move faster when the compiler is strict, because I don't have to think about whole classes of errors—they're automagically checked.
I like my compiler to be disciplined, so I don't have to.
Still, for example - I can see that dynamic types reduce a certain quantity of boilerplate. If you just don't make the kind of mistakes that make static checking necessary, then you could be more productive that way. I think a lot of boilerplate stuff is a bit like that.
I wouldn't be - but still.
Boilerplate isn't the reason why static typing might be worse than dynamic typing. Undecidability is. Inference algorithms tend to reject some correct programs. The real question is then, are we interested in those programs? So far, my experience has been "not really". I know of a few cumbersome exceptions, but overall, Hindley-Milner type inference basically lets me write the programs I want.
It's not like Java is a worthy ambassador for static typing.
I know it's not as much of a big deal in Haskell where mutable state is limited, but I still remember it being an annoyance to have to manually breakfast out my data to disk and then reload it again.
A good example is "variadic" functions. Consider computing the max of three numbers. Clojure's `max` takes any number of arguments, while Haskell has `max` that takes two arguments, or `maximum` that takes a list of arguments; both require some boilerplate to apply if you have exactly three.
It's impossible to write down a Haskell function equivalent to Clojure's `apply`. You have to work around its absence, e.g. with awkward folds.
Or for a more immediate example, compare Clojure's flexible `map` to Haskell's big 'zip' family: zip1, zip2, zip3, zipWith5...
The whole concept of an `apply` function is a bit alien to me. I generally just call the damn function. Also, Haskell has an `apply` function. It's the `$` operator, defined thus (the first line is optional):
`$` :: (a -> b) -> a -> b
f $ x = f x
f $ x $ y
Well, you have to select one of the zip functions in addition to map. Those are more names.
> Also, Haskell has an `apply` function. It's the `$` operator
That isn't what apply does. Apply takes a function and applies it to a list of arguments, (f x y) is equivalent to (apply f (list x y)).
And the way to translate that in Haskell is to apply a single argument.
Haskell functions are not Lisp functions. Lisp functions take a list of arguments, so `apply` should apply to a list of argument. Haskell functions have one argument, so `apply` should apply to a single argument. That argument could be a tuple, by the way.
I understand how currying works, thanks, but that doesn't change the fact that $ is not equivalent to apply. A version of apply that dealt with curried functions would just have to fold over a list, and I don't think it could be properly generalised in Haskell's type system (but I"m not an expert at Haskell).
For your definition of apply, Lisp-1's don't have it, because as you said, you'd just call the damn function. Lisp-2's sort of have it, but it's called funcall. Regardless, your definition of apply is not what apply is in Lisp.
You are seeing that a square hole isn't fitting a round peg, and accusing the hole of being too sharp. Or at least millstone did. Complaining about the absence of a Clojure like `apply` function in Haskell doesn't make any sense.
It would be like complaining about having to work around the lack of infix notation in Forth. Whoever does that need to let go of their old thinking habits, and rewire their brain to the new language. Not everybody can do that. Fewer still want to do that.
Now if someone says, "this real world problem is easier to solve in Clojure, in Haskell you have to jump through hoops", that would be different. For the present case, one would have to show how the absence of a proper Clojure like `apply` function could force anyone to jump from any hoops to solve any concrete problem.
> Regardless, your definition of apply is not what apply is in Lisp.
You've either said too much, or not enough. What "apply" does mean in Lisp, then?
apply :: Dynamic -> [Dynamic] -> Maybe Dynamic
apply fn args =
foldM dynApply fn args
max [a, b, c]
But there are many language level tools that will help with your productivity. Some are all upside things, others come with different trade-offs. The article isn't even focused on compile-time verifications.
But that was at a time when more people learned Lisp, there was less choice in tools and the eco-systems were smaller. In the 70s/80s one could buy ten years into the future with the right hardware/software and government/military was financing.
Take for example the Connection Machine CM 1, an early massive-parallel computer with 2^16 processors. It was initially developed largely for and with Lisp. You could program it in *Lisp from a Lisp Machine - one of the most expensive co-processors. Fortran and C was added then for certain commercial users. Very expensive stuff and at least ten years ahead.
Today the landscape looks different.
The Objectstore database was developed by former Lispers, who wrote an earlier object-oriented database in Lisp.
You can bet this was simply due to lack of Lisp developers at Yahoo. And I can bet that the Lisp code was easier to read.
I once delivered a sophisticated pricing modeler to a financial institution, in Python, done mostly in functional style. Code well documented and commented, and easy to understand. And Python is one of the easiest languages to learn.
The customer's IT deparment insisted on a complete rewrite to Java, because that's what their developers knew.
This was safer for pointy haired bosses :)
I think this says less about the language, and more about the kind of talent you will attract based on your language choice.
However the full iceberg is still only found on the Lisp languages.
Some features are found in other languages, but the combination of metaprogramming with homoiconicity is one of the key elements, and that is arguably only found in Lisp.
In the Common Lisp world there is Quicklisp which works perfectly, however "nano libraries" a la NPM are often shunned in that community (as well as in many others).
Often a Common Lisp library will only require 2 or 3 dependencies and each one might require 0, 1, or 2 dependencies.
Part of this is due to the comprehensive "standard library" built into the language specification itself, part is also due to maturity -- some third party libs have become de-facto standards.
No, ASDF and Quicklisp do different things. ASDF handles compiling and loading systems, whereas Quicklisp handles downloading systems and their dependencies. ASDF is kind of like make, and Quicklisp is kind of like a package manager.
Since we are in 2018, I'll go with python