Hacker News Reader

Comment by obl | Hacker News Reader

Since this is intented to be educational I find it a bit unfortunate to implement function calls using the host (python) stack since it's an important part of an interpreter.

It means that you can't really add things like stack traces, coroutines, exceptions, single stepping, etc and understand how they work under the hood.

View on HN

Not sure why you say that: I know Chakra uses the native stack for both its interpreter and JIT and it obviously has stack traces and debugging. It's possible to walk the stack pointer chain and mark explicit frames as host or guest frames

So does V8, albeit with a different calling convention than the surrounding C frames.

Isn't this how Python itself implements function calls, using the host (C) stack? Hence Python's lack of support for stackful coroutines?

Aren’t Python stack frames allocated on the heap, like everything else?

No. You can try this at home:

  import sys
  sys.setrecursionlimit(2 ** 32 - 1)
  def recurse():
      recurse()
  recurse()

This code attempts to create unboundedly many Python stack frames. If those frames were allocated on the heap, you would expect it to either throw an exception or else display symptoms of memory exhaustion. Instead, it segfaults.

Doesn't the fact that python by default has a recursion limit imply its at least doing a little extra work?

Yes, but the point is that that extra work is still happening on the stack, not on the heap. Overflowing the C stack results in a segfault; exhausting heap memory doesn't.

When it performs a Python call, CPython performs a corresponding recursive call of the interpreter function. It uses both the C stack to manage interpreter state and heap-allocated stack frame objects to manage the interpreted program's state.

Your experiment shows that the C stack overflows faster than the heap. It does not show that no heap space is consumed at all.

The recursion limit exists simply to prevent the user-level code from exhausting the native C stack.

On the other hand CPython in fact does extra work because it also creates heap allocated frame objects (which essentially mirror the C stack) for debugging purposes.

> Doesn't the fact that python by default has a recursion limit imply its at least doing a little extra work?

No? Pretty much the entire point of the recursion limit is to protect against segfaults.

Perhaps a topic for future blog post. I considered having virtual frames but thought it would obscure the point of the post, which was the relationship between compiler and interpreter.

Reply on news.ycombinator.com