Hacker News Reader

Comment by tneumann | Hacker News Reader

I am the the original author, and trust me, I am describing a real world problem. I run massive parallel data processing tasks on machines with 128 cores, and unfortunately some of them produce errors deep within the processing pipeline. From a programming perspective exceptions would be ideal for that scenario, but they cause severe performance problems.

Just think about this: If you have a 100 cores, and 1% of your tasks fail, one core is constantly unwinding. And due to the global lock you quickly get a queue of single threaded unwinding tasks. And things become worse, we expect to have machine with 256 cores soon, and there it is even more dangerous to throw.

If you do not believe me look here: http://wg21.link/p0709 It lists quite a few applications that explicitly forbid exceptions due to performance concerns.

View on HN

> and 1% of your tasks fail

That's a lot of failures. I'd question whether or not you should be using exceptions for something that happens 1% of the time all the time.

Admittedly you might not have any choice in the matter if it's a dependency that is failing.

The "exceptions should be exceptional aka rare" line sounds aspirational, but I wonder how true it is if you survey the general landscape of actual libraries.

On top of that, you never know the context in which your users will call your library.

Are you encoding a single file on a local machine? Or 1000 files on some HPC 128-core machine?

Do we really want to be in a world where all libraries need to be written with and without exceptions since we can't know the context they'll be used in?

Why is the error occurring in the first place? In my opinion, exceptions/errors should be rare. If it's a software problem, it should eventually be fixed. If it's an infrastructure problem, it should be fixed. It's not just something that should continue to happen without any mitigation what so ever over time.

Not all errors have "fixes". If I navigate to "https://news.ycombinator.com/nope" it returns an error to me. Is that a software problem or an infrastructure problem? And what's the fix? How do you "fix" a bad input to the system from an external actor?

So no, errors/exceptions are not rare depending on the context. So now do you use exceptions and accept that it doesn't scale across core counts? Or do you use return values and accept that it just makes all your code slower in all cases?

1% error rate sounds like a lot. But really how many HTTP errors do web servers return on average? Or other such systems? It's definitely more than 0%, and those services also scale nicely across multiple cores, so... Now we're just bikeshedding over the example numbers, not the core issue.

You are not wrong. The exceptional exception guideline feels circular. I would use exceptions significantly more if I could rely on their performance.

Probably not that rare to use exceptions but I imagine it is very rare to use exceptions for normal flow control - that is, to continue execution after an exception. It's a common pattern in Python but not C++.

I've done it exactly once and I ended up removing it because it makes debugging exceptions that you care about really annoying.

> but I imagine it is very rare to use exceptions for normal flow control - that is, to continue execution after an exception

Exceptions are at least a very central part of the normal flow control of parser combinator libraries.

I agree with the premise - C++ exceptions are not a good choice for signalling numerical errors in high-performance computing environments, where said errors can happen with some frequency.

However, I do not agree with the conclusion - what is the difference between removing exceptions and simply not using them in your project? All compilers already let you compile code without exceptions and the product I work on compiles all numerical code that way. I don't see what benefit you expect to reap from changing the fundamental way C++ exceptions are implemented and work. At the high end, you want a custom error solution fit for your particular task, I don't think we can have a generic exception framework that will work for every high-performance computing project.

I've worked in low latency trading, and with large distributed back testing setups with hundreds of high core count machines, and not once felt the need to disable exceptions, or for that matter, felt the impact of them happening. A fair bit of noexcept stuff was useful to improve runtime performance, but that was about it.

I would suggest you need to address that 1% of failing tasks and determine what the issue is, as frankly, you are solving the wrong problem. If I might suggest a solution, it sounds like you are using threading when process farms might be a better solution.

(and if i'm way off the mark with my suggestion, apologies, i'm trying to help and have little information to work with).

Even reducing it to .1% we'll still have 1000 cores or at least threads before too long. GPUs already have thousands. Prosumer CPUs already have 128 threads. Epyc supports SMT, and will be 512 threads on dual socket Bergamo in 2023 I think on the roadmap. There is an intermediate at 384 threads dual socket Genoa in 2022.

Low latency is not the issue per se. It’s more like being able to guarantee a worst case upper bound runtime. Maybe your system can tolerate indeterminate but rare worst case run times but many useful systems cannot. E.g. flight control systems but also ideally any interactive system.

Well i'm now working in a hard realtime domain (audio, noone dies when it goes wrong, but it's still hard realtime) and i've still got exceptions turned on. But no exceptions are thrown during runtime, they're there for exceptions. Yes, there's overhead in places, but we noexcept some code paths to reduce that overhead in some very hot code paths.

If you allow exceptions in your real-time audio loop then when an exception occurs the only useful thing to do would be to take down the entire process. Handling exceptions as a norm in your audio loop would be unacceptable because it could never be done in a way that would always meet the deadline. In that case, if you cannot usefully handle the exception, why have your exceptional situation cause an exception in the first place? You’re just as well off calling abort().

> If you allow exceptions in your real-time audio loop then when an exception occurs the only useful thing to do would be to take down the entire process.

as a user of audio software, I would really prefer them to have the occasional glitch instead of crashing, potentially loosing work. Like, how it is even a discussion ?

Sure but that isn’t the issue. We’re discussing the architecture of an audio loop, e.g. handling exceptions as a norm in the codebase. If it is avoided then there is no large behavioral difference between calling abort() and throwing an exception.

... I hope I just don't understand. Say you have some audio node in your engine which at some point fails because of some error condition, for instance it tried to load a preset in an invalid format which was supposed to be sanitized beforehand, the preset tried to load a sample and the OS returned -ENOPERM, whatever.

Surely you agree that it's better to have this throw an exception, and a big try.. catch around your audio callback so that just this tick, which loads the preset, fails, and then the program continues normally at the next tick, just with the wrong preset for a node, rather than calling abort() which entirely kills the program and looses what the user was working on ?

> Surely you agree that it's better to have this throw an exception, and a big try.. catch around your audio callback so that just this tick

Yes I do agree but my point is that if any exception is being thrown for the sake of being handled then your Audio code is bad.

Generally your Audio loop should encounter no errors at all (it should be a pure process) so what the OP is saying is that excepting/aborting is fine in that case because it would truly be exceptional. I can’t imagine what the exceptional situation would be but to the extent one exists, I agree with that. My only point is that throwing an exception when there is no handler is no better than calling abort(), obviating the need for enabling exceptions.

> Yes I do agree but my point is that if any exception is being thrown for the sake of being handled then your Audio code is bad.

thank god ! then we both agree.

> Generally your Audio loop should encounter no errors at all (it should be a pure process)

and audio handlers should never lock mutexes yet I have seen my fair share of them in the wild.

> My only point is that throwing an exception when there is no handler is no better than calling abort()

every thread or event loop should always have a catch-all handler at top level.

> and audio handlers should never lock mutexes yet I have seen my fair share of them in the wild.

They are incorrect. Only bounded-time synchronization primitives can be used. E.g. lock-free queues.

> every thread or event loop should always have a catch-all handler at top level.

Two things. 1) There should be no reason for a properly coded real time audio loop to have a top level handler because exceptions should not be thrown because they cannot be guaranteed to be handled within the necessary time constraints.

2) top level exception handlers divorced from context cannot be handled, only ignored. Ignoring exceptions is very dangerous because there is generally no way for the top level handler to be sure that the rest of the program is in a correct state (unless it knows all code is exception-safe, which is rare). Continuing to run can result in data corruption. Exceptions leaving the context that has the information to handle them is a programming error, just like a failed assertion.

> They are incorrect. Only bounded-time synchronization primitives can be used. E.g. lock-free queues.

yes, my point is that incorrect code exists in the wild, that's the world we have to live in.

> properly coded real time audio loop

most things aren't properly coded, and yet still useful

I see a difference between bugs and design flaws.

I'm not sure I understand the appeal of C++ exceptions. For years I've been told that Rust and Go's lack of advanced exception support is a crutch; and that error returns or sum types were unimaginative.

Now, in this thread, exceptions are supposed to be used rarely. I don't see the difference between an exception and Rust/Go's `panic`. If the error is rare enough, then chances are you cannot gracefully recover. If the error isn't rare; then why are you using exceptions for control flow?

Exceptions have always supposed to be used rarely - that's the point! The most obvious example is 'new' failing and returning std::bad_alloc. Given that new is implicit all over the place (pushing to a vector for example) the code would get very cluttered if you had to cope with this everywhere, so the pragmatic decision was taken to use exception handling to give apps that cared the chance to deal with them, and those that don't a sensible error message by default when the exception escapes main.

It's interesting that the lack of enforced exception handling was seen as a strength when exceptions were added to the language, compared to now when the lack of enforced exception handling is sometimes seen as a weakness. Programming language design and expectations are a moving target, so it'll be interesting to see how these things develop over time.

And yes, Rust and Go take a different approach. Is there right and wrong? No, i'd say just different approaches which will play to the strengths of some problem domains and coding styles.

> If the error is rare enough, then chances are you cannot gracefully recover.

that does not match my experience. Generally, rolling back to the state at the beginning of whatever user interaction caused the exception is good enough in a large amount of cases and a much better experience for end-users than abort()

Not only is this relevant to HPC environments, it also impacts the other end of the spectrum in the embedded space. When dealing with real-time requirements, you are much more concerned with the worst case performance as opposed to the average case or happy path performance.

This article makes it very clear that exceptions complicate analysis of performance. The non-local nature of exceptions mean I can't analyze performance in isolation. E.g. I can test that thread 1 always meets its deadlines (even with exceptions being thrown), then I can test that thread 2 always meets its deadlines.... but if thread 1 and thread 2 happen to throw exceptions at nearly the some time I might miss both deadlines. Who knows, maybe this means a thruster fires too long and that insertion burn fails ... and Mars has a new crater instead of a lander.

And, once you throw -fno-exceptions you are no longer using standard C++, which the standard library assumes. So, using anything that would throw exceptions on memory allocation failures is a no-go. You can work around this with extensive use of allocators (that reference enough static memory to avoid any possible out-of-memory situation)... but this is not looking like idiomatic C++ anymore, and most off-the-shelf libraries are unusable.

A completely local exceptions implementation (e.g. Herbceptions) would solve this.

Using local allocators where they offer some benefit certainly is idiomatic C++. Freeing all objects so allocated by simply reclaiming the blocks from a local allocator is also idiomatic C++.

Major subsystems that use no memory except what is passed in from above is also idiomatic C++.

C++ is a big tent. Things you do routinely in one part of a program, such as at startup, may be very different from what you do in a main loop, or in termination cleanup. Things my program does may be very different from what your program does.

> I am the the original author, and trust me, I am describing a real world problem.

for a more-or-less niche part of "real world". The overwhelming majority of desktop GUI apps rely on some C++ system - Qt, gtkmm, Wx, Blink, Gecko, FLTK, etc etc... and it is not an issue for those, for what exceptions are commonly used for (a write failing because the user disconnected the USB drive while it was copying, a system resource limit exhausted.. things like that). As much as massive parallel data processing tasks matter, I'd really prefer my language to not side-step writing end-user apps for something that happens at $bigcompany or $bigresearchlab.

here's the list of binaries that link against libstdc++.so.6 in my /usr/bin: https://paste.ofcode.org/gfZJwx4puVx7Uxy9a7BBU3 - don't forget those please :)

Are you arguing that C++ is unsuitable for massive parallel data processing tasks?

I think he's arguing that C++ exceptions are unsuitable for parallel data processing, at least in cases where exceptions are regularly thrown.

If exceptions are regularly thrown the software has a bad design and must be fixed. Non-exceptional stuff must of course not be handled through exceptions - no exception should ever be thrown if the software operates as it is expected to.

There is the red flag: "regularly thrown". "Regularly" is opposite to "exceptionally".

Well of course, they are slow so they are unsuitable. The issue is whether they need to be slow.

No, he's arguing that he'd prefer it not be if that means breaking support for end user apps which is what the OP is arguing for.

No, of course not. But let's not change the language in ways that benefits those use cases at the detriment of more common use cases.

Of course, if there are ways to keep more or less the same semantics, while increasing performance, by all means it should be done !

for ones that throw an exception for 1% of a unit computation, perhaps yes.

But as has been noted, that's a lot of failures and exceptions may not be the appropriate mechanism to deal with this type of failure.

Indeed, arithmetic errors are what NaN was invented for. It still works, and is fast.

You mention the problem of having to recompile for ABI-breaking improvements. Just out of curiosity - in the environment you're describing, how much of the code cannot be rebuilt? As I was reading I was assuming that most situations where exception handling causes performance issues are probably also situations where you have most if not all the source code available. This wasn't the case in my formative C++ years when 3rd party libs were expensive and distributed as binaries (on floppies).

> I run massive parallel data processing tasks on machines with 128 cores

So, not to be rude here, but this may be a real-world scenario in your world, but not in everyone else’s. I get the sense the kind of stuff you’re working on would benefit from things like using assembly as well. But for desktop apps, games, compilers, other command line tools… who cares about the performance of throwing exceptions?

Games cares about it. But you're right that it's only very high-performance contexts.

All of this hinges around the global lock, although I'm not sure what you mean exactly. Are you talking about memory allocation, a lock that exception handling takes or something else?

The main problem is the unwinding lock. Memory allocation is a bit unfortunate, too, but much less so.

Note that I am trying to fix the unwinding problem, I have submitted a patch to libunwind to eliminate the contention during unwinding:

https://reviews.llvm.org/D120243

It require application support, unfortunately, as there is currently no way that libunwind can figure out if a shared library has been added or removed. But if you are willing to indicate that from within the application the performance problem is mostly fixed. The memory allocation issue remains, but I can live with that. I cannot live with single threaded unwinding.

Thanks for that patch - I'll watch this with great interest.

Is there a problem if dynamic libraries invoke dlopen/dlclose they also need to call the sync function - correct?

I'm asking because we've developed a Common Lisp implementation that interoperates with C++ and it uses exception handling to unwind the stack (https://github.com/clasp-developers/clasp.git). We hit this global lock in unwinding problem a lot - it causes us a lot of grief.

If you are seeing 1% of operations failing, I can guarantee you are Doing It Wrong.

If you run into contention issues have you tried scaling horizontally? Running on 128 single core VMs should mitigate what you see.

That would turn communicating between threads into communicating between VMs, which is a disastrous change in itself.

It would. On the other hand multiprocessing with explicitly shared memory could be a fast (but potentially painful) workaround.

That could make sense for processes, which would also get around the exception unwind lock, but I don't know how that would work with every thread moved into a separate VM.

Of course. Using VMs here would very deep into overengineering territory.

how do you know the threads need to communicate with each other?

How do you know they don't?

If you want to avoid a global process lock, multiple processes will have less orchestration and communication overhead than multiple VMs (though still a lot more than necessary with threads within a process).

Reply on news.ycombinator.com