Hacker News Reader

Comment by peterkelly | Hacker News Reader

Have you ever

- Traversed a directory hierarchy to compute the total size or other statistics about its contents?

- Searched for a DOM element on a web page matching certain criteria without using third-party libraries or helper functions like querySelector()?

- Implemented a hierarchical navigation structure in a user interface?

- Work on parts of a compiler/interpreter or other code that has to deal with formal language?

- Written a parser for a file format that contains arbitrarily nested data structures (e.g. HTML/JSON)?

- Done anything with trees?

- Solved a problem by breaking it down into smaller versions of the same problem, solving those, and then combining the results?

I know that for lots of programming tasks recursion is not needed, but I'm genuinely curious as to what kind of software you work on where you've never encountered a problem that requires a recursive solution?

View on HN

I've been working mostly in the payments domain for the last 16 years. 90% backend, 10% web frontend.

So far, I've had write recursive code just once to convert SOAP requests/response to REST request/response on the fly so that the test cases that were written using SOAP client could be reused to test REST end point as well.

To state the obvious, the chances of encountering recursive code depends on the domain. In my experience though a typical CRUD business app won't require a recursion as you pointed out. The examples you sited are mostly encountered in a "framework" code (e.g., Spring, some UI framework) and as is typically the case the number of developers "using" a framework is an order of magnitude more than those who implement/maintain them. Same is the case with compiler. On the other hand, those who deal with the lower level system code (OS Kernel, RTOS etc.,) shun recursion altogether to avoid its unpredictable stack need. So in the spectrum of developers the band of coders who frequently deal with or encounter recursion is quite narrow IMO.

There are languages where recursion is the only construct available to deal with a collection of values so for those programmers recursive technique is a muscle memory. But I suspect there aren't as many professional users of such languages.

> I know that for lots of programming tasks recursion is not needed, but I'm genuinely curious as to what kind of software you work on where you've never encountered a problem that requires a recursive solution?

There are no problems that require recursion, since recursion and iteration are equivalent. You can do all of the things you list with a while loop plus state variables.

Recursion (especially in languages with pattern matching and other features common in the same set of languages in which recursion is more idiomatically favored) can provided solutions that are simultaneously more clear and more terse, but it doesn't expand the set of solvable problems.

Let's improve peterkelly's question. How about:

I'm genuinely curious as to what kind of software you work on where you've never encountered a problem where a while loop required an ad hoc dynamic stack to track state, and so never had to consider the possibility of recursion as a viable, attractive alternative to that.

Using a stack-based data structure of your own is always better than using a call stack for recursive traversal:

- You get to store only the state you strictly need and no more. A call stack stores all local variables and parameters for each frame, potentially wasting memory.

- You get to access previous elements. A call stack does not typically allow you to access data in previous stack frames.

- You get to push/pop multiple, variable number of elements per iteration. Call stacks typically only work with a fixed number of local variables/parameters per stack frame. The workarounds are to 1. dynamically allocate memory for each traversal, which causes lots of fragmentation, or 2. use an oversized fixed buffer for each stack frame, which causes memory waste, or 3. use an ‘alloca’-like construct, if your runtime supports it at all.

- You get to explicitly bound its memory usage and use custom handling/error reporting when your desired max depth is exceeded. Doing so with a call stack is typically a lot more clunky and bug-prone, as it requires exception handling and mutating global state.

- You can serialize/deserialize your stack to/from persistent storage however you like if you wish to pause traversal and resume it later. A call stack can only do this if it’s a ‘stackful coroutine’, which even if it is supported in your runtime of choice, gives you little to no control over where or how it is stored.

None of the above actually require recursion.

Some of the above problems have very neat recursive solution but the recursion is bad idea as it tends to various edge cases.

For example, "write a parser for a file format that contains arbitarily nested data strucutres", is exactly a problem where you DO NOT want to use recursion, and the reason is that "arbitarily" part. What if there is 1000 levels? What if there is 100 thousand levels?

DoS guys love these kinds of implementations. Just send very small, very well compressed payload and look how it all crashes and burns.

You’d run into the same denial of service problem in an iterative solution with an explicit stack/data structure that can be grown arbitrarily as well. The only way to prevent such an attack is an explicit limit (perhaps calculated at run time based on system specs) which can be done with either a recursive or iterative implementation but requires deliberate awareness either way.

Fair point, but how frequently do you see recursive implementations that also pass information to track recursion depth and various other information to limit resource usage?

Have you ever seen a recursive call that also tries to detect/break cycles in the data structure?

I suspect the reason these things get left out the moment the developer decides on a recursive function is because it doesn't look nearly as much elegant then.

Depth is just an extra parameter d, which you pass to the recursive calls as d + 1. This is still elegant compared to tracking depth in a non-recursive solution.

When you add this parameter to a recursive solution that is lacking it, your compiler will flag all the places where you are neglecting to pass it, and you just copy and paste d + 1 into those places. Or at worst, it will be caught as a run-ime insufficient parameters error.

At the end of this exercise, you not only have working code, but an inductive proof that d has the correct value everywhere. There are no cases where the depth is mismanaged, as could happen in some iterative solution where you must insert assignments to d, that do not create any sort of error when they are missing.

I would say, DoS people love these kinds of specifications.

I would specify the data structure is ostensibly arbitrary, but conforming implementations can diagnose and reject more than 64 levels of nesting.

Anything needing more than 64 levels of nesting can safely be assumed to be DoS.

Recursion doesn't use more space than iteration in languages with tail call optimization

That's false.

You would be correct if you wrote "Tail-optimizable recursion doesn't use more space than iteration in languages with tail call optimization"

You're right, that's what I meant.

:(){ :|:& };:

Never once had to do any of those in a professional setting. Mostly just creating reading, updating, and deleted database records with a very simple UI (if any at all).

I have used recursion, particularly due to my love for Erlang, just not in any software someone paid me to work on.

Reply on news.ycombinator.com