Hacker News Reader

Comment by mpweiher | Hacker News Reader

> Keep in mind that a compiler like SBCL will put Lisp performancewise in the range of other statically compiled language/implementations.

Hmm...that claim is made a lot, and it works, in theory (see Jitterdämmerung[1]). However practical reports from non-advocates don't seem to bear out those results. For example, Michael Stonebraker says that they started writing Postgres in LISP. Had to revert that decision as everything was an order of magnitude slower[2].

[1] http://blog.metaobject.com/2015/10/jitterdammerung.html

[2] Michael Stonbraker's Turing Award Lecture: https://youtu.be/sEPTZVGk3WY?t=1791

View on HN

> started writing Postgres in LISP. Had to revert that decision as everything was an order of magnitude slower

If they have never written a high-performance application in Lisp, they would have needed a bit more work. The Lisp version was written more as an experiment:

http://db.cs.berkeley.edu/papers/ERL-M90-34.pdf

> Our feeling is that the use of LISP has been a terrible mistake for several reasons. First, current LISP environments are very large. To run a ‘‘nothing’’ program in LISP requires about 3 mbytes of address space. Hence, POSTGRES exceeds 4 mbytes in size, all but 1 mbyte is the LISP compiler, editor and assorted other non required (or even desired) functions. Hence, we suffer from a gigantic footprint. Second, a DBMS never wants to stop when garbage collection happens. Any response time sensitive pro- gram must therefore allocate and deallocate space manually, so that garbage collection never happens dur- ing normal processing. Consequently, we spent extra effort ensuring that LISP garbage collection is not used by POSTGRES. Hence, this aspect of LISP, which improves programmer productivity, was not avail- able to us. Third, LISP execution is slow. As noted in the performance figures in the next section our LISP code is more than twice as slow as the comparable C code. Of course, it is possible that we are not skilled LISP programmers or do not know how to optimize the language; hence our experience should be suitably discounted.

They had problems because Lisp alone needed 3 Megabytes of address space (!) to run - a 'gigantic footprint' - , needed to work around the GC and their C version was twice as fast.

Their main problem was that they wanted to write it in a mix of C with Lisp and it was hard to debug for them. I'm not surprised that people with no experience writing applications in a mix of C + Lisp failed to write a database engine. I'm pretty sure it is easier to write a database engine in C compared to a mix of C and Lisp, without GC and crossing the boundaries of a managed runtime and plain C data. People did that, but these people had quite a bit more experience writing such applications.

You can buy for example AllegroGraph, a graph database in Allegro Common Lisp:

https://allegrograph.com

You can bet that the NSA/CIA/Homeland Security etc. all have it running with very large datasets about you, me and all the others.

From the section you quoted:

"As noted in the performance figures in the next section our LISP code is more than twice as slow as the comparable C code."

I don't doubt that you can write productive systems in CL (bit of a fan myself), but the "oh we have smart compilers that just make it as fast as C" just isn't true.

> I don't doubt that you can write productive systems in CL (bit of a fan myself), but the "oh we have smart compilers that just make it as fast as C" just isn't true.

But 50% of C is great. Imagine in Lisp you can achieve 50% performance of C. That means you can write large amounts of your code in Lisp, without the need to use C.

Actually it is both better and worse. One can achieve better than 50% and in many cases one is slower than 50% of C. For example you can optimize your code at runtime using the Lisp compiler. But in many cases you will be slower, since C compilers and libraries tend to have better support for hardware.

For many many applications 50% performance of C is completely sufficient. Zillions of lines of code are written in much slower environments...

It must not be forgotten that performance isn't always the main concern, let alone the only one.

And yeah, the expressiveness/speed ratio of Lisp is great.

I don't deny it, just that that expressiveness has a cost and Lisp advocates assume that cost is negligible, while in reality it is important(Lisp Machines were created just so that Lisp would run at acceptable speed)

Lisp Machines were developed as a new breed of computers: personal workstations for AI researchers.

Starting in the 80s a lot of people/companies made Lisp run with 'acceptable speed' on more conventional Personal Computers, once these got available.

The thing is, back in the 80s (before I was even born!), the cost was huge, but it has stayed the same until now. So, yeah, it cannot best good C/C++, or even optimized Java, but it still outperforms any scripting language.

You know what they say, the right tool for the right job.

> But 50% of C is great.

But 50% (2x) was the lower bound of the slowdown, it was at least twice as slow (from that article), and in the Turing Award lecture Stonebraker mentions an order of magnitude.

Once again, I agree with most of your points, I just think the "we have magical compilers that make the overhead go away" (paraphrasing) is a bit too glib.

50% of speed isn't the whole deal. Latency and memory use matter too. Audio,Video, Games, Networking,etc require minimal latency. Memory use is also critical: when you data structures explode into GBs of RAM, a twice larger means you can run out of RAM and start swapping to disk, lowering performance substantially. Finally 50% of speed in isolated benchmarks with minimal GC use doesn't mean 50% inside a complex application, where GC can pause the current thread or steal time continuously.

I have a couple of Lisp Machines at home. They ran an object-oriented OS fully written in Lisp incl. window system, networking with TCP/IP, object-oriented database, text editor, graphics editor, 3d suite used by companies like Nintendo and Sony, mail server, file server, ...

That thing ran on a 5 MIPS (Millions instructions per second) 40 bit processor with 40 MB RAM in the end 80s / early 90s.

Suddenly with 1000 times more CPU speed, multi-core machines, fast GPUs, GBs of RAM, etc. it should not be possible to do that anymore?

Because suddenly Lisp is too slow? Why was it fast enough to run that stuff in the 80s on tiny machines?

Naughty Dog wrote Crash Bandicoot for the Playstation using a Common Lisp IDE and a specialized Scheme runtime for the Playstation. It sold millions of games and critics liked it for its fluid gameplay.

Suddenly with machines much faster than the first Playstation this should no longer be possible?

>Why was it fast enough to run that stuff in the 80s on tiny machines

Lisp Machines were invented precisely because Lisp was slow on common hardware, hence the 8 extra tag bits. Lisp Machines are specialized Lisp hardware.

Lisp Machines were invented as the first personal workstations, because the common hardware was timeshared PDPs. The researchers wanted powerful personal computers with interactive user interfaces - something which did not exist at that time.

Still that was not the question. The question was why 1000 times faster computers with much more space should not be able to run Lisp applications when in the 70s it was possible to run a whole OS on tiny machines.

I don't claim that "Lisp cannot run on modern hardware", just that it has performance sapping features that lower its competitiveness vs mainstream compiled languages. If you don't care about performance and don't pretend to be in the same league as C i have no problem with Lisp/Scheme. Its the same deal with people pretending languages with GC such as Java are fit for systems programming and low-level tasks.

Yet most enterprise servers are written for GCed infrastructure: .net and JVM.

Actually Lisp runtimes perform not worse than the JVM.

Java pretty much defeated the whole "the sky will fall if we deploy something with GC" nonsense. (Thank you, Java). What you're dealing with in this thread is twenty-year-old recycled FUD.

"Ruby proves its systems programming language as fast as C because i deployed an app on rented server i know nothing about"

I've taken a fair crack at it, but I'm unable to find an interpretation of your sentence such that it relates to my comment.

Your Apps (such as Java) are user-level programs. Its irrelevant if you can run Interpreted Brainfuck or Python on Crutches. Read this comment: https://news.ycombinator.com/item?id=14488427

These servers run C-based OSes where Java/.Net is just a server app.

I knew that.

But you ignore the whole stack the OS, Libraries, the Java runtime all are written in C/C++ only the end-user server app is written in Java code(that obviously means C/C++ is obsolete and Java is the systems language as fast as C/C++).

Why do you think I ignore that and what does this have to do with this discussion?

"Yet most enterprise servers are written for GCed infrastructure: .net and JVM." Link to comment: https://news.ycombinator.com/item?id=14487354

That's even worse, there are other layers below the GCed Java/.net infrastructure, which makes the whole thing even slower and less performant.

That basically means Lisp is probably about as fast as C because: C that is portable, easy to maintain, and robust against bad inputs also only runs at 50% of the speed of C.

If you take any fast C program and add all the logic that is missing to make it robust and secure, and refactor it so that it is halfway nimble against changing requirements, it's no longer running at the mythical speed of C.

Reply on news.ycombinator.com