Hacker News Reader

Comment by chalst | Hacker News Reader

I don't consider a language supportive of functional programming unless it supports tail-call optimisation, which is performed by some but not all implementations of Common Lisp.

The article recommends SBCL, which does support TCO.

An old (2011) survey of TCO support is at https://0branch.com/notes/tco-cl.html

View on HN

More context:

SBCL supports TCO: see http://www.sbcl.org/manual/#index-Tail-recursion

One can drive whether SBCL optimizes tail calls by setting the proper optimization settings: see http://www.sbcl.org/manual/#Debugger-Policy-Control

That sounds more like an implementation detail, in most cases. By this metric Haskell wouldn't be a functional language.

It's a very important implementation detail. Explicit loops tend to imply mutation, which is contrary to idiomatic functional programming. Recursive calls don't require mutation but do require TCO to achieve equivalent space complexity. Constant-factor optimizations are one thing but failing to perform TCO turns constant-space algorithms into linear-space algorithms (or linear ones into quadratic, etc.). It's less a matter of "optimizing" the calls and more a matter of not wasting limited stack space on data which is clearly not required to execute the remainder of the program. One might as well label the practice of freeing stack frames when a function returns "Function Return Optimization" (FRO) and consider it a mere "implementation detail". After all, wouldn't it be much simpler to grab new memory every time the program needs some storage space and never bother with cleaning it up? It would certainly make debugging easier with all those old variables retained for the life of the program and not constantly overwritten by new data. However, programs written for a language without guaranteed "FRO" would look very different from normal programs, much as programs designed to compensate for the lack of guaranteed TCO look very different from idiomatic functional programs.

Haskell uses a different (data-centric, non-strict) evaluation model where recursive definitions don't result in recursive calls, so traditional TCO isn't as relevant. Recursion is used very heavily in Haskell—which has no first-class looping constructs—but the resulting programs generally do not require large stacks. It's not unusual to be able to run even large Haskell programs with a 1 KiB maximum stack size (+RTS -K1k). Space leaks are possible, of course, but they take place in the heap.

Common Lisp was designed without requiring TCO because:

* various platforms don't support TCO. It was designed such that it can be implemented by a simple non-TCO interpreter, transpiled to a non-TCO C compiler, compiled to a non-TCO Lisp Machine CPU, or to a non-TCO virtual machine (like the JVM). Many languages don't support TCO on the JVM and may only implement explicit tail recursion or have a compiler detecting tail recursion - which is far from supporting TCO. Thus a portable conforming Common Lisp program will run in ABCL (a full Common Lisp implementation on top of the JVM) - because it will not depend on TCO to not blow up the stack, or similar.

* another reason was that Lisp has a bunch of features with don't work that well with TCO. For example Lisp always supported various dynamic scoping constructs and made extensive use of those - something which Scheme in its core does not, but provides via libraries or language extensions. Using dynamic scoping constructs makes TCO more difficult, may require a different language design, etc.

I agree with you on the first point. There are good technical reasons why TCO can't be implemented on some platforms. However, that just punts the issue one level down the stack: These platforms should have built in support for guaranteed TCO.

As for language features like dynamic scoping, I would say that a function call which needs to be followed by some cleanup activity is not in tail position, so TCO would not apply. The cleanup code could be in tail position, however, if implemented as a function call. In Common Lisp most forms of dynamic scoping or unwinding are explicit anyway, so this shouldn't come as a surprise as it might in languages like C++ and Rust where destructors are called implicitly when objects go out of scope.

They are often explicit, but they are widely used and often generated behind the scenes by macros or declarations.

Macros would need to specify whether any expressions will be evaluated in tail position. However, expressions in macros don't look like they should be subject to TCO, so the default assumption should be that they aren't unless declared otherwise. Do you have any examples of cases that would be likely to cause confusion—in particular where a function call appears to occur in tail position in the code but can't be TCO'd because of a macro?

For example I see sometimes macros which generate code with compilation quality (speed, ...) declarations for all or parts of their code. Depending on the combination of qualities TCO might be enabled or disabled in code sections.

If TCO is guaranteed at the language level, as in Scheme, then it will always be enabled regardless of compilation settings. Debug builds are no more tolerant of stack space leaks than release builds. The fact that TCO isn't guaranteed is the problem here.

> If TCO is guaranteed at the language level, as in Scheme, then it will always be enabled regardless of compilation settings

https://www.gnu.org/software/kawa/Compatibility.html

Your point? The page you linked to specifically says that Kawa only implements a subset of modern Scheme—by which I mean R5RS or later. Early versions of the Scheme language standard didn't require TCO, but all the recent ones do. This doesn't affect the core point that if TCO is guaranteed by the language standard, as in modern Scheme, then it cannot be selectively disabled because doing so would break perfectly compliant code.

Haskell implements call-by-need reduction, in which all calls are tail calls.

Indeed, tail-call optimization is not part of the standard, unlike Scheme.

The point of the article is that while the Common Lisp standard is not really focused on functional programming, nothing prevents the implementions (and libraries) of today to be so.

All the CL implementations that matter support TCO.

Reply on news.ycombinator.com