Hacker News Reader

Comment by smueller1234 | Hacker News Reader

I thought the choice to implement locals and all of their semantics using just the main stack was both curious and surprising. (I'm most familiar with perl's implementation, which requires a bunch of stacks flying in for formation to implement it's complex scoping semantics.)

In the book, Bob says he designed the language such that it would be possible to use just one stack. What does that mean? It sounds very discouraging for newbies because it implies a lot of foresight or trial and error.

Overall, while this is among the most interesting subjects covered, I find the chapter a bit lacking in either one of two ways: either it is for beginners, and then the chapter feels very magical or is for intermediates and then a bit of extra verbiage on the design choices that led you there would be appropriate.

Nitpick: I would totally have stopped low and done the -1 initialization trick as well. But yuck. ;)

View on HN

The constraint on the language for this to work well is fairly natural and is actually described in the article: the language should follow lexical scoping.

Unfortunately, since some dynamic languages do not respect lexical scoping, programmers in these languages tends to think of local variables and scoping as something very complicated. It doesn't need to be.

You can have plain lexical scoping but still be unable to use a simple stack in all cases, due to closures.

The author describes this additional constraint that they have on top of lexical scoping.

> We have to be OK with only allocating new locals on the top of the stack, and we have to accept that we can only discard a local when nothing is above it on the stack.

Well, there are lot's of well known compiler techniques to handle closures in that context. It's a little bit out of scope of the article, but it's not very complicated either.

I know that but you said the constraint needed was lexical scoping - that constraint is insufficient and you need additional constraints on the language design.

If you’re happy for the compiler to copy variables out of the closed-over scope when returning the closure, this can still be handled with a single stack. That’s what C++ lambdas do: “Closures” in C++ are locally-allocated structures that hold used variables as local (stack-allocated) member variables. Creating a closure copies closed-over variables (or pointers/references to them). Returning a closure from a function returns a logical copy (which can be optimised away) of the structure.

(Don’t get me wrong, this obviously still implies additional constraints, but it gets fairly close to universal closures.)

I don't really understand that point of view - you can use a single stack as long as you actually use the heap in addition to a single stack?

C++ closures do not, by themselves, use the heap. Every closure in C++ gets translated by the compiler to a unique type that contains either copies of or references to the objects being closed over. If you choose to use copies, then whether or not anything is allocated on the heap depends on the copy constructor; if you use references then there is no copying, but it's up to you to ensure lifetime.

This is roughly how Java works with anonymous types closing over variables also. That's why the variables must be declared `final`. It just copies the local values over into the anonymous type and calls it a day. Of course, since the only thing allocated on the stack are primitives and pointers, and everything on the heap is subject to garbage collection, this is a pretty straight-forward operation.

I don't know if lambdas work the same way. I know in some ways they work like anonymous types, and not in others.

yes, it works the same way with lambdas, the lambda proxy (the class that implements the functional interface) contains the copy of the local values.

Here is the code that generate the constructor of a lambda proxy http://hg.openjdk.java.net/jdk/jdk/file/3cabb47758c9/src/jav...

C++ lambdas do not use the heap. As I wrote, they are stack allocated.

You're talking about undefined behaviour - this is a badly broken feature of C++ because it does not move the local variable it references onto the heap or some other alternate data structure.

Look up the upwards funarg problem.

> The upwards funarg problem arises when the calling function refers to the called/exited function's state after that function has returned. Therefore, the stack frame containing the called function's state variables must not be deallocated when the function returns, violating the stack-based function call paradigm.

C++ violates the constraint we're talking about.

No, I’m not talking about UB. Although you’re right that if you use references rather than copies and return a lambda then, yes, you’ll run into UB. The solution to this is to use copies, as mentioned.

> Look up the upwards funarg problem.

I’m well aware of upward funargs and what I’ve described specifically implements upward funargs in C++. See https://godbolt.org/z/KZiBNB. Note that this code does not perform any heap allocation, and the closed-over variable `i` is saved from the local scope and made available to the caller via the lambda.

> Although you’re right that if you use references rather than copies and return a lambda then, yes, you’ll run into UB. The solution to this is to use copies, as mentioned.

That was the context I thought we were having the conversation in - capturing local variables. I'd describe what you mean as capturing local values. If you're copying the value from a variable then you're not capturing the variable.

I can see why you're arguing it your way now.

> it does not move the local variable it references onto the heap

C++ doesn't do such thing automatically, but there's nothing preventing you to do it yourself. If you intend to use a local variable whose lifetime would not normally outlast the closure, feel free to use std::move and a move constructor. In general C++ gives you the tools to manage memory manually or semi-automatically, but not totally automatically. And it also doesn't, unlike Rust, force you to use memory correctly.

Note that in chapter 3.8.1 he writes "As you can imagine, implementing [closures] adds some complexity because we can no longer assume variable scope works strictly like a stack where local variables evaporate the moment the function returns. We’re going to have a fun time learning how to make these work and do so efficiently."

Reply on news.ycombinator.com