Current hardware trends make C++ exceptions harder to justify
open-std.org
open-std.org
Exceptions are exceptional so in principle it doesn't matter (within reason) how long it takes to throw one as long as it costs nothing not to do so. So measuring the cost of repeated throws IMHO doesn't cast light on any useful case, and the approaches that add runtime cost for the path not taken, even Herb Sutter's, are not acceptable.
His code transformation example is simply the compiler behaving properly, unless it can't make that transformation even when foo() is declared noexcept.
The high core count is a real issue and a legitimate reason for an ABI break. Exceptions are the kind of below the surface plumbing that can't reasonably be implemented in regular code.
And in that regard the paper does make a good suggestion, though it then dismisses it! The tree approach described in section 3.4 seems like the right kind of fix.
Ultimately there's a spectrum of branching (`return` -> `break` -> `goto` -> `throw`) all of which need consideration in light of multicore deployment.
I deplore that - its side-effects writ large. But those that use it, find it reasonable and sensible.
It's become hard to distinguish right and wrong when it comes to CPU optimization. Different versions of the 'same' CPU can have wildly different sweet spots.
Indeed, though I am shocked when people post benchmarks in which they are obviously unaware of this issue. It's why compilers have all those architecture switches.
If I understand your comment correctly: in the case of the posted paper, the compiler optimization issue the author mentioned was a semantics issue, not an architecture issue. If your comment was about optimizing for the multiprocessor case, I apologize.
Oh for the good ol days when register size was all you had to think about!
The problem isn’t that C++ exceptions are slow when used the way the compiler expects. The problem is a mismatch between how C++ expects you to use the feature, and how people are actually using the feature.
I can’t find the story now, but I read about this happening with Ruby on Rails. Someone dug into why rails was so slow at Twitter, and found the way rails was looping through an array was that it would loop unbounded, and catch and discard the array out of bounds exception at the end of the loop. Whenever Ruby throws, it allocates 10k of memory and fills it with all sorts of information for debugging - including a full stack trace. And it was doing this work in the background every time someone iterated through an array. Fixing this resulted in a massive speed up at Twitter, and presumably across the entire rails ecosystem.
I saw the same thing at (big tech company) a decade or so ago. I was working on a project which used GWT. We brought some people in to help us optimize, because the program was too slow. The first thing they found was that the JS VM was spending most of its time in the exception handler for some reason. Turned out one of our engineers had a habit of using throw & catch as a way to do multi level returns in complex code in Java. But the exception was being converted to a javascript exception by GWT. And javascript exceptions are (were?) super slow.
Y’all gotta stop using exceptions like that. I know it feels clever. But in most programming languages, using exceptions for control flow will kill your performance.
Jesus
Great detail I'd forgotten: At the time twitter took 460ms of compute to process each request. Almost all the CPU time was in bcopy, constructing the big string backtraces for exceptions (that were then being immediately discarded).
I think the approach Go, Rust and others take there is the right way to go. By making panics annoying or even impossible to recover from in some cases, they ensure that they are only used for their intended purpose and are not abused as a general purpose control flow mechanism.
We can't just blame the programming language for programmers not understanding how computers work.
I think as computers have gotten faster, and languages higher level, we've stopped talking about computers as mechanical devices. And this is a really important perspective to have.
Can you answer these questions about your program?
- How big is your binary / JS bundle? What parts take up most of the space?
- When your program runs, what does the computer spend most of its time executing? What parts of your program are the slowest, and why?
- How big is the memory footprint? Which parts of your program use the most RAM?
For binary programs (like rust / Go / C), which patterns are easier or harder for the optimizer?
If there are two ways to design your code, how do you discover which approach will run faster?
This stuff shouldn't be considered advanced concepts. An architect understands the building they've designed. A chef knows what their food tastes like. When you program, you're making a thing. You should understand what you made and how it will be executed on the computer.
shudder The worst of cargo cults!
A design pattern is a way of communication. If you turn it into a procrustean tool you don't even understand why they exist.
Imagine writing a parser. Can you use exceptions? It's hard to say! In fact, often impossible to say! How often an exception is thrown is highly dependent on the input data. If you're in a controlled environment with nearly always well formed data, it might be a perf win! But there's a ton of complexity in just deciding which type of error to use, and thats on top of the complexity of now having multiple ways of returning errors!
No one likes code that has 2+ ways of indicating method failure. If you are a library author, do your users need to check for both Exceptions and Error codes? If you are a nice author, you'll just choose 1 way to indicate failure and wrap where necessary.
If you want to use exceptions, they should be used in very limited scope when perf measurements indicate they would help, tightly wrapped in a catch, and then immediately converted to some other standard error type.
Even better, we should just let the choice of when to use exceptions up to an optimizing compiler, and provide the compiler a function to convert between exceptions and our more general purpose error type of choice.
I am aware there are functional languages that have other ways of dealing with this, or languages with things like optional values etc. But fundamentaly I like the idea that there's just a mechanism to just collapse, and collapse in a way that is traceable.
Having said that, my experience essentially does not involve catching exceptions, to me that's what is almost unthinkable (read: probably a bad hack). I guess my use case is not performance but separateing normal code function from catastrophic errors. Could you explain, do you think this is a fundamental mistake in the way I'm approaching this?
Which is part of the problem. If you're not catching the exception, you don't need exceptions. You just need a way to log stuff, such as a trace, and exit the program. New languages such as Rust and Go encourage this explicitly - that is, use the ordained error path in the return value for expected errors, and only panic for catastrophic errors (or use panic in hotpath where return overhead has unacceptable performance impact and errors are super rare).
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p070...
> Programs bugs are not recoverable run-time errors and so should not be reported as exceptions or error codes. — We must express preconditions, but using a tool other than exceptions... migrate std:: away from throwing exceptions for precondition violations.
Exactly. But the upstream caller of your function might need it.
And that’s the beauty of it. If I don’t know how to handle a particular error in one context, I can just leave it and let it propagate upwards.
For example downloadFile()->parseHttpResponseStrean()->readSocket()
If readSocket throws a timeoutError I don’t need every http library to explicitly deal with it. What can they even do about it, apart from retrying? The application level code calling the downloadFile function is still free to catch the error and gracefully show an error to the user or use a default file from disk if they please.
Very few languages are as finely tuned to the needs of libraries and users of libraries as C++. Exceptions are an essential part of that compact. Failing to abide by it makes your library a bad citizen.
Yes + syntactic sugar. Propagating an error up can and should be a single character of code.
> collapse in a way that is traceable.
Zig supports "error traces" (note, not stack traces) by default, since they have a single common way of reporting errors and a smart compiler.
> I like the idea that there's just a mechanism to just collapse,
Go uses Panic + Recover.
The overlap between "things that fail and the caller knows something useful to do", and "things that fail and somebody maybe six levels back in the call chain might know something to do" is remarkably thin. Most things are the latter. Once in a while, it makes sense to provide one for each: (1) open a file, and it had better succeed or you might have to bail out, or (2) see if a file is there.
The reason exception is the default, aside from that there are plenty of ways to return an error report, is that when you need code to clean up, the destructors are right there. They get exercised every time through the code, so are well-tested. How often is your special-snowflake error handler ever exercised? And the next one? And the next one? Even before you have given a moment's thought to error handling, your destructors are cleaning up, correctly. So, even if you haven't written code to handle it, your program has remained in a well-defined state. And, very often, you don't even need to write them: they are auto-generated from things the compiler already knows.
If you have set up a high-level catch block, you get there without fuss, and that code knows something correct to do, even if it is only to shut down cleanly. If you left a record of what was about to be attempted, that code knows what to try next.
My point is exactly that. It's more important to have a single common way of doing error handling than it is to have a small perf improvement. That's why I recommend return values. You'll never hit a perf problem with them.
Exceptions cannot be used in code hot paths (~thousands of iterations) when it's not a small perf problem anymore. Therefore, they cannot be your only method for signaling failure. And therefore, assuming we care about consistency and composability of code, it's far better to use return value based error handling unless you have good reason (almost never). Ideally with syntatic sugar from the compiler (see Rust, Zig, Swift, etc).
> when you need code to clean up, the destructors are right there.
Exceptions are not needed to achieve this. C++ calls the same 'well tested' destructors when you return from a function.
The one point in your favor, that I acknowledged above, is that C++ requires exceptions to report failure from Constructors and Destructors, for OOP, std::vector initialization, for much of the standard library etc. Thats a travesty.
You are arguing that programmers should not be obliged to make judicious choices. But that is exactly the job of every programmer. Dodging consequential choices practically defines a bad programmer.
Your assertion that destructors handle errors reported by return value is disingenuous, if not actively dishonest. The code checking for and acting on a failure is not in a destructor, and is never exercised except when you succeed in provoking exactly that failure.
You are writing bad code, and should be ashamed.
> Exceptions cannot be used in code hot paths (~thousands of iterations) when it's not a small perf problem anymore.
You should read the paper in the link. Both of your claims here are exactly backward.
The problem is "just" that exceptions don't scale across multiple threads due to internally using a global lock. They are otherwise much faster than return values (4x faster in the trivial example)
I disagree with this. I think they should be used whenever "standard" return value error handling starts to obfuscate the normal operation (happy flow) of the code. This impact on developer performance happens a lot sooner than the measurable machine performance.
In rust, propagating an error up is as easy as adding a '?' after the function call.
fn read() -> Result<String, io::Error> {
let f = File::open("hello.txt")?; // propgate up?
let s = f.read_to_string(&mut s)?; // propgate up?
Ok(s)
}Turns out I strongly prefer explicit errors over exceptions, I just also strongly prefer error handling not consuming 75% of the lines on my screen.
And in the cases where I really do prefer a panic, there's always `.unwrap()`
The serialization of stack unwinding issue is a real serious problem.
Well yes, but EH is basically trying to be the best way to do a highly non-local goto with a dynamic target. If that part is your bottleneck than indeed you may have a more specialized requirement.
I do on occasion build a homemade version of a standard library datatype because I have a specialized need and only need implement the subset our code will call. That doesn't invalidate the more general implementations in the standard library (which tend to be quite good, even corner cases).
Likewise sometimes I check for a null pointer being returned or other local error flag. That doesn't mean I don't think exceptions are a good idea.
> The serialization of stack unwinding issue is a real serious problem.
That is the important part of the paper and as I wrote in my comment, I am disappointed that a possible solution, written and tested by the paper's author, was dismissed.
I think it could be done by changing the mangling algorithm (basically: compile fails if new-ABI versions are not available) but I haven't thought enough about it to remove the words "I think" from the beginning of the sentence.
In any case this doesn't seem to be an ABI issue
If you write a jpeg-processing service, it’s intuitive to raise an exception on a malformed jpeg, but there’s no guarantee that only 1% of the jpegs users upload to the service are malformed.
In other words, we treat exceptions as exceptions from our code expects, not what’s statistically unlikely in the input space, which is in many cases impossible to predict with accuracy. (E.g., even if only 1% of your inputs are malformed over all time, on some Thursday you may be hit with 80% bad inputs, making the performance drop across the service unacceptable.)
Not really. Having to parse a malformed doc is not an exceptional situation: it's a basic use case, and one which is very close to the happy path.
Which the paper covers, that's the std::expected section: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p254...
As a bonus not only is using exceptions for this faster, it's also less error prone. There's much less risk of any intermediate function or helper abstraction failing to propagate an error code properly.
Well, it would be if exceptions weren't avoided like the plague in C++ because they have a, well, bad implementation, which can't be meaningfully fixed without ABI breakges (as the paper covers)
Which really should be possible with exceptions. It's a shame that it isn't.
So a function that ingests a file and processes it may throw an exception if the file isn't found so that the UI can catch it and ask the user for an alternative filename (or to give up and not open a file at all).
If you're connecting to a remote machine and don't get a response, you might throw an exception because you don't know if the user typed the name wrong.
While if you are already talking to a machine and it stops responding it's reasonable to wait a moment and retry, as if could be a transient network brown-out which is something you can deal with on your own.
In a situtaion where are more bad jpegs than good ones, we still don't necessarily care that they cause the slow path, if the purpose of the service is doing meaningful processing with good jpegs.
This is in contrast to say python where the convention is to rely heavily on exceptions as part of the normal flow of the code, sometimes described as "asking forgiveness, not permission". It is not uncommon for a method argument to support multiple types, and to discern them by treating the object like one type and if you get an exception, then try treating it like another type. Likewise, if you're not sure an item is in a dict, you just try to access it, and catch the exception if it isn't. This has performance impacts, but so does everything else about python, so it isn't worth optimizing.
Where I do think this pattern makes sense is in trying to use system resources. Checking that a file exists or a process is alive before deleting or killing it is a recipe for difficult to track down Time of Check to Time of Use bugs. I'm curious if you think this is also an anti-pattern in c++ and if so, how you properly deal with the TOCTU race conditions?
If the library you are using does throw exceptions, then I would probably start with just using exception handling. Then if I notice poor performance, add an initial check, purely as an optimization, while keeping the exception handling to deal with race condition.
So if you are trying to hold a lock you don't throw an exception, you just wait and try again. Perhaps you can't reach that host; try again a few tiles before giving up and throwing an exception. But if you try to write to removable media and the device won't open, all, your program isn't going to mount a tape itself: throw an exception and let the problem be handled at a higher level.
I would have not have thought an unreachable host was exceptional, given that it's quite normal.
The point is that handling the host unreachable is almost always semantically higher level than the code opening the connection. This the code opening doesn’t implement the policy of what to do when the situation occurs. In the case, say, of a hard-coded address then it’s possible the author of that piece of code does know how to handle a host unreachable case.
Btw, we don't say "exceptions are for infrequent conditions", because that's not what they're for.
I quite like the etymological "taken out", because it carries a notion of special handling - a control flow aspect that is the main point of using them.
> Traditional C++ exceptions have two main problems:
> 1) the exceptions are allocated in dynamic memory because of inheritance and because of non-local constructs like std::current_exception. This prevents basic optimizations like transforming a throw into a goto, because other parts of the program should be able to see that dynamically allocated exception object. And it causes problems with throwing exceptions in out-of-memory situations.
> 2) exception unwinding is effectively single-threaded, because the table driven unwinder logic used by modern C++ compilers grabs a global mutex to protect the tables from concurrent changes. This has disastrous consequences for high core counts and makes exceptions nearly unusable on such machines.
That's really interesting. I've become very anti-exceptions in recent years far various much-discussed reasons (eg hard to follow, false economy, difficult if not impossible to write threadsafe C++ code in particular, use of exceptions as flow control is an anti-pattern).
One of the porposals is a value-or-error type object, which is basically what Rust has. I really like Rust's enums and match expressions.
It seems so difficult to make changes like this to C++ at this point, at what point do you just have to start again?
It leads to the current situation where you have C++ "the language" which is everything, and then C++ "the subset that everyone uses" where that subset constantly changes with time and development context.
But yea this kind of thing hit me recently on the interview circuit. I wrote some correct C++ (in that it was correct for the idioms in vogue when I last wrote C++ regularly for money) but I got feedback I wasn't as senior as they hoped due to my (lack of) C++ knowledge. Part of that was a shitty interviewer, but it's also just a fundamental part of the language. If you leave for a few years or try and change shops you find that everything under you has been removed or a completely different subset of the language is being used elsewhere. The complete lack of an ecosystem just reinforces that.
It doesn't happen only in C++ circles.
While C++'s churn frequency seems to be higher than the Javascript front end churn frequency, as an outsider, it still seems like "best practices" on C++ are churning around 1.5-2 years, it's been happening for my entire career, and it's still happening. If I seem a bit unsympathetic to the claims that the problems are solved if you just write your C++ code this way now, it's because I first heard that in 1998 or so, for a set of common practices now considered laughably out of date, of course.
At some point it becomes more cost-effective to just "churn" on to Rust next time, because even though in that same time frame Rust is a younger language that was going through its early design iteration phase it still seems like it has settled into a lower-frequency churn rate lately for "best practices" than C++.
There's probably some interesting and deeply profound reason why C++ just can't seem to stabilize across what is approaching an entire human generation, but I'm nowhere near interested enough in learning it to actually learn the amount of C++ it would take to find it.
Rust is still adding a ton of new language features, especially around async, compile time code evaluation and the type system (const generics, GAT/HKT, existential types etc.). We'll very likely see further developments in more areas next, e.g. to match C++ developments in parallel and heterogenous compute (GPU's and the like), or to add forms of proof-carrying code, etc.
I have to disagree with this quite strongly. What "best practices" churn are you seeing? Your reference example of "don't use new & delete" (which I agree with) was a single best practice change that happened ~10 years ago. Similarly https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines is like 5-7 years old now, and afaik hasn't had any significant revisions?
The "churn" was really pre-C++11 to post-C++11. It was more like a python 3 moment than churn, other than it's taking a long, long time for code bases to catch up.
It's not like C++ is oscillating. It continues to improve, which is a good thing.
There are great uses and great features, but there are so many of them and everyone has their own opinions.... Even in this thread there's a clear subset of people who "adore" C++ exceptions
And I would have to say the average C++ dev did not know about RAII in 1993.
But what exactly is wrong with that? I don't quite understand your argument here..
Prolly not, I expect the issue comes from the increase in branches since a value-based error reporting has to branch on every function return. Even if the branch is predictible, it’s not free.
And fib() would be a worst-case scenario as it does very little per-call, the constant per-call overhead would be rather major.
The intended use-case of catch_unwind is to protect the wider program e.g. avoid breaking a threadpool worker or a scheduler on panic, or transmit the information cross threads or to collation tools like sentry.
A problem with using exceptions instead is they defeat the return address prediction by unwinding the stack.
For the record, I like absl::StatusOr<>.
As the article explains, the problems in C++ are more an ABI issue than a programming language issue (except for the by-reference vs. by-value semantics). You could implement exceptions internally by variant-like return values, for example, similar to how error passing is done in Rust, while still having it look like exceptions on the language level. It would be fun for future languages and runtimes to more easily be able to switch the underlying mechanism, or possibly to be able to use different implementation mechanisms for different parts of a program as needed.
Especially when used as an error ADT they're awful because they mix everything from "you have to handle this 90% of the time" to "the universe physics just changed, sorry" into one construct. Much better to use something like Vavr's Try and explicitly propagate the error as a value.
[1] https://www.artima.com/articles/the-trouble-with-checked-exc...
If I declare "throws NullPointerException", then this should mean I want it to be a checked exception. This should force the caller to catch the exception, or declare throwing it, or to simply throw it out again without declaring it. This would effectively convert the checked exception into an unchecked exception.
Converting a checked exception to an unchecked exception is possible in Java, and I do it whenever it makes the most sense instead of wrapping the exception, which is just a mess. Unfortunately, there's no way to make an unchecked exception behave as if it was a checked exception. It's easier to reduce fault tolerant checks then to add more in.
Some might argue that converting an exception to be unchecked is a bad idea, but this sort of thing is done all the time in JVM languages that are designed to interoperate with Java. If I call a Scala library from Java, and that library is doing I/O, then IOException has now effectively become unchecked.
I understand your point about usage-dependend checking. However, I believe it is mistaken. Consider a call chain A -> B -> C -> D, where D throws a checked exception of type X, which C converts into an unchecked exception (still type X). At the same time, B also calls other methods that happen to throw a checked X, and thus B throws a checked X itself, which now, unbeknownst to B, also includes the X from D. B documents the semantic of its X exception, which may not fit the ones thrown by D. Now A catches X, believing the semantics as documented by B, but actually also catches X from D, which was concealed (made unchecked) by C. This breaks checked exceptions in their role as part of an interface contract.
The fact that unchecked exception may originate from arbitrarily deep in the call chain is also the reason why they are unsuitable for defining interface contracts, A function declaring certain semantics for a particular unchecked exception can't realistically ensure those semantics if any function it calls itself must be assumed to also throw exceptions of that type (because it is unchecked). Effectively, you can't safely document "exception X means condition Y" for your function if any nested calls may also throw X for unknown reasons.
If you have a function that has an abnormal result as its API, it should have that as part of its return value, because returning as values is what you do with results. Checked exceptions in contrast don't compose. See for example this code:
X myFunction(Y param) throws MyFailureException();
List<X> transformed = list.stream()
.map(e -> myFunction(e))
.collect(toList());
Instead, if you make the part of the return value you can do whatever. I'll use Vavr Try as an example, but you can also do this in Java 17 with sealed interfaces or in a miriad other ways.
Try<X> myFunction(Y param);
List<Try<X>> transformed = list.stream()
.map(e -> myFunction(e))
.collect(toList());
That is, the checked part is just a language implementation, right? The jvm doesn't really make much of a distinction. (This is meant as a check to my assumption.)
As always someone is wrong on Internet.
Java adopted a feature that was initially introduced in CLU, adopted by Mesa/Cedar, Modula-2+ and Modula-3, was being considered for ongoing ISO C++ standardization at the time.
By people who do not want to believe that errors are part of a system's API and prefer to just write the happy path and let any exception kill the process. And who don't mind getting called at 2:00 am because a dependency buried deep in a subsystem threw an exception that you'd never heard of before.
This is a really interesting and nuanced point here. The article linked above [1] talks a bit about it. The problem is, they both are and aren't part of the API in the strict sense.
In the sense that they specify a contractual behaviour they are part of the API of a function. But in the sense that they are something the caller should / needs to specifically care about, they sit in between. That is, in the vast majority of cases, the caller does not care specifically what exception occurred. Generally they want to clean up resources and pass the error up the chain. It is "exceptional" that a caller will react in a specific way to to a specific type of exception. So this is where Java goes wrong because it forces the fine grained exception handling into the client when the majority case (and preferred case generally) is the opposite. It makes you treat the minority case as the main case. There are ways to work around / deal with this but nearly all of them are bad. The article talks about some of the badness.
I do think it's interesting though that Rust has taken off and is generally admired for a very similar type of feature (compiler enforced memory safety). I am really curious how that will age, but so far it seems like it is holding up.
[1] https://www.artima.com/articles/the-trouble-with-checked-exc...
Case in point, the many checked exceptions in the Java standard library that are never, ever thrown. Ever. Such as ByteArrayOutputStream#close(). In fact the docs on that method even say it doesn't do anything, and won't throw an exception. But there's still a checked exception that you have to handle, because it came from the interface.
Which is part of why checked exceptions are a mistake. If Java wasn't so aggressively OOP, then maybe there's a good idea there. Like maybe checked exceptions in C++ would work, as you're not relying (as much) on inheritance to provide common functionality. But as soon as interfaces & anonymous types (eg, lambdas) enter the scene, it starts falling over.
Also in terms of API design, checked exceptions are quite limiting. Especially when working with asynchronous APIs, as checked exceptions are inherently coupled to synchronous calling conventions.
And there's also then the problem of not a whole lot of your code base is an "API surface" (hopefully anyway), and it's really vague how a middle layer should handle checked exceptions. Just propagate everything? But that's a maintenance disaster & leaks all sorts of implementation details. Just convert everything to a single type? Well now you can't catch specific errors as easily.
The point was that you should be returning errors, not throwing them. Runtime exceptions (null reference, division by zero, out of memory, etc.) ought to indicate a fatal error in the (sub)program or runtime environment. You can trap these, and report them, but it's usually a mistake to try to case-match on them. Unlike errors, which are predictable, enumerable elements of the design, runtime exceptions should be treated as an open set.
I think this paper started with a conclusion, and worked backwards to justify it.
Just think about this: If you have a 100 cores, and 1% of your tasks fail, one core is constantly unwinding. And due to the global lock you quickly get a queue of single threaded unwinding tasks. And things become worse, we expect to have machine with 256 cores soon, and there it is even more dangerous to throw.
If you do not believe me look here: http://wg21.link/p0709 It lists quite a few applications that explicitly forbid exceptions due to performance concerns.
That's a lot of failures. I'd question whether or not you should be using exceptions for something that happens 1% of the time all the time.
Admittedly you might not have any choice in the matter if it's a dependency that is failing.
However, I do not agree with the conclusion - what is the difference between removing exceptions and simply not using them in your project? All compilers already let you compile code without exceptions and the product I work on compiles all numerical code that way. I don't see what benefit you expect to reap from changing the fundamental way C++ exceptions are implemented and work. At the high end, you want a custom error solution fit for your particular task, I don't think we can have a generic exception framework that will work for every high-performance computing project.
I would suggest you need to address that 1% of failing tasks and determine what the issue is, as frankly, you are solving the wrong problem. If I might suggest a solution, it sounds like you are using threading when process farms might be a better solution.
(and if i'm way off the mark with my suggestion, apologies, i'm trying to help and have little information to work with).
Now, in this thread, exceptions are supposed to be used rarely. I don't see the difference between an exception and Rust/Go's `panic`. If the error is rare enough, then chances are you cannot gracefully recover. If the error isn't rare; then why are you using exceptions for control flow?
This article makes it very clear that exceptions complicate analysis of performance. The non-local nature of exceptions mean I can't analyze performance in isolation. E.g. I can test that thread 1 always meets its deadlines (even with exceptions being thrown), then I can test that thread 2 always meets its deadlines.... but if thread 1 and thread 2 happen to throw exceptions at nearly the some time I might miss both deadlines. Who knows, maybe this means a thruster fires too long and that insertion burn fails ... and Mars has a new crater instead of a lander.
And, once you throw -fno-exceptions you are no longer using standard C++, which the standard library assumes. So, using anything that would throw exceptions on memory allocation failures is a no-go. You can work around this with extensive use of allocators (that reference enough static memory to avoid any possible out-of-memory situation)... but this is not looking like idiomatic C++ anymore, and most off-the-shelf libraries are unusable.
A completely local exceptions implementation (e.g. Herbceptions) would solve this.
Major subsystems that use no memory except what is passed in from above is also idiomatic C++.
C++ is a big tent. Things you do routinely in one part of a program, such as at startup, may be very different from what you do in a main loop, or in termination cleanup. Things my program does may be very different from what your program does.
for a more-or-less niche part of "real world". The overwhelming majority of desktop GUI apps rely on some C++ system - Qt, gtkmm, Wx, Blink, Gecko, FLTK, etc etc... and it is not an issue for those, for what exceptions are commonly used for (a write failing because the user disconnected the USB drive while it was copying, a system resource limit exhausted.. things like that). As much as massive parallel data processing tasks matter, I'd really prefer my language to not side-step writing end-user apps for something that happens at $bigcompany or $bigresearchlab.
here's the list of binaries that link against libstdc++.so.6 in my /usr/bin: https://paste.ofcode.org/gfZJwx4puVx7Uxy9a7BBU3 - don't forget those please :)
So, not to be rude here, but this may be a real-world scenario in your world, but not in everyone else’s. I get the sense the kind of stuff you’re working on would benefit from things like using assembly as well. But for desktop apps, games, compilers, other command line tools… who cares about the performance of throwing exceptions?
Note that I am trying to fix the unwinding problem, I have submitted a patch to libunwind to eliminate the contention during unwinding:
https://reviews.llvm.org/D120243
It require application support, unfortunately, as there is currently no way that libunwind can figure out if a shared library has been added or removed. But if you are willing to indicate that from within the application the performance problem is mostly fixed. The memory allocation issue remains, but I can live with that. I cannot live with single threaded unwinding.
I only use exceptions for cleanly unwinding the stack when an unrecoverable error occurs. I design and implement my code so that is rare.
Code that uses exceptions like this is a code smell, especially when performance is important. Using exceptions as control flow instead of if/loops is not a good design, IMO.
Note that this is in C++. I consider this style of writing not idiomatic, despite the STL doing it. I use either error return codes or std::optional (itself having a set of problems but IMO better than exceptions).
I'm more willing to accept this kind of code in Java, where it's more or less idiomatic. Less so in C#, where the tendency in the last 15 years has been to use the Try__ method instead of throwing exceptions.
You could argue we were using "too many exceptions", but to me it's the obvious way to unwind in C++.. except it's not fast enough, so we had to switch to a proto-Rust (this was before Rust) style system.
My argument against exceptions is basically everything other than performance. To make a performance decision you have to build the code incrementally with feedback from how it is actually going to be used e.g. if almost every parse fails then you probably don't want to throw whereas if your code almost always succeeds you probably want your register back (i.e. use exceptions)
The articles suggestion that it's only a matter of time for 128+ core CPUs to be common, at least in the server space, is probably the least contentious argument it makes?
You can get a machine with dual 64-core AMD EPYC 7662 CPUs[0] for a total of 128 cores. It will cost you almost $20k, though—for the most basic configuration.
[0] https://bizon-tech.com/bizon-x6000.html#959:8714;960:4858;96...
In particular he has already responded to the efficiency argument with the counter argument that current implementations can be optimized. Even if that optimization process breaks ABI compatibility, it’s still better than breaking source compatibility, which is usually what is being proposed. He is right about that so I don’t think efficiency arguments are going to sway him.
I used to embrace C++ exceptions until I had to use C++ in non-conventional environments. For me C++ exceptions are wholly inappropriate for real-time programming since it’s difficult to statically quantify how much time an exception handling sequence may take. There’s also the issue of it requiring malloc() which has its own issues from an interface standpoint in the real-time context. To avoid unbounded malloc, you’d have to set aside a per-thread area for exception storage and require that you never throw an exception value past a certain size.
1) you're describing a fairly unusual application behavior 2) it's only because of the performance goals that the speed of exceptions matters
I'm the lead dev of a cross-platform DAW where our "inner loop" is real-time constrained by hardware. We use exceptions freely (in a multicore, multithreaded context) with no performance issues, because if they ever happen, things must stop.
I keep reading how terrible exceptions are, and the examples are almost always in hot inner loops. If a junior dev put this code in a PR (a senior dev should NOT be doing this), we would require them to fix it.
Your argument seems to be that "well, yeah, of course exceptions should suck, you shouldn't ever use them" and just... being happy with that? The purpose of the paper is to say "hey, maybe exceptions shouldn't be a dumpster fire piece of hot garbage?"
I reserve exceptions for a really exceptional situations.
My motto is that if an exception happens, the program should crash since it's an unrecoverable error. That's just my opinion by I try to enforce that in my own codebase.
My main beef with exceptions is that sneak past the type system. Unlike Java with its exception specification, in C++ we have no idea what to catch. Yes, there's documentation, but it's often hard to keep it in sync with the code. I've been bitten many times in the last couple of years by surprising exceptions jumping from deep inside innocent function calls, either because of a deep dependency or because someone just threw one and didn't document it (that someone was also a younger me on occasion...)
So my design criteria for exception is, is this error here unrecoverable for the application?
The 2 examples in OP's paper are 2 functions, and we have no context. If they were a part, say, of a console application that was used in a nightly cron job, and there was no user to ask for proper input, then yes, maybe crashing the application is acceptable. I would still have liked to print something to a log.
If this was a part of an interactive program, then a proper error code / std::optional / other error object about the illegal value and its place in the array should be returned, with a proper error message displayed to the user about logged. This I'd do by having a top-level catch handler. A single one for all the application.
If I'm writing a string_to_int function today I would return an optional<int> (or some Either variant). But if exceptions were cheap I would use them. If I'm catching an exception in the context of immediate caller and the compiler inlines the calle, it ought to be able to optimise the exceptional path away. But it doesn't happen.
no, not any errors, exceptional errors. Input data coming from "outside" being invalid is not exceptional. The filesystem suddenly becoming inaccessible during a copying or caching operation, the OS not being able to join a thread, your logging backend becoming unable to log because the disk is full, or a regular expression being incorrect are what comes to mind when I think of exceptional situations (unless your program allows users to input regexps). If your program runs on a normal computer in normal conditions, no exceptions should ever be thrown.
And in any case this is about how these exceptions impact parallel processing - and the impact is not negligible.
The problem is in that case you get into a probabilistic estimation, which is a nice way to say you roll the dice on your performances: what’s the ratio at which exceptions are too costly or sufficiently cheap? And how does that impact your level of service if e.g. exceptions are extremely rare but they tend to all affect the same request or workload?
Because the discussion is about methods of reporting and handling failure?
> The only reasonable "probabilistic estimation" is that something that happens 10% of the time is normal and it shouldn't be treated as an "exception"
That makes no sense whatsoever, and doesn't address the question.
And even if a ctor fails at a rate of 0.9, it has to report errors via exceptions, because that's the only mechanism available to ctors.
Or I think my preference would be a private constructor, and a friend-function that returns value-or-error.
Chromium code replaces exceptions with boolean flags and logging or code to kill the current process for bad cases like out-of-bound access.
I don't use them. I just create an error type and pass that around.
The only legitimate exception I will accept is when you access invalid memory. That's a special case and depending on the environment something extraordinary must happen.
But exceptions and exception handling just creates annoying code. It doesn't add value, not really.
1. They generate syntactic noise at every point they touch the call graph: function signatures, calls, returns.
2. In particular, if a change causes a deeply nested function that used to always succeed to be able to error, the entire path up the call graph needs to get Resultified.
3. Since the caller must be aware of them, generic code generally has to be Result-aware too.
4. They aren't a total solution, exceptions usually have to exist anyway (eg panic), for things like oom/assert/etc. So you're usually paying the cost of them anyway.
5. You only get what the callee gives you. With exceptions, you can get a backtrace to the cause of the error by default, with no effort needed on the part of the callee.
This is a pro not a con. It now shows clearly that all these function calls are now failable. Of course at higher levels you may have additional assumptions that you know won't make the low-level functions fail, and you are welcome not to change everything as well.
> 4. They aren't a total solution, exceptions usually have to exist anyway (eg panic), for things like oom/assert/etc. So you're usually paying the cost of them anyway.
Panics do not need to be caught and handled. You can (and should) transform panics into abort.
I think this strong stance needs some justification, especially since it's not the default in Rust.
This is not "noise" but needed information. A function that can error out should not have the same signature as one that will never return an error. Similarly, call-site special syntax (like '?' in Rust) helps address the concerns raised by hidden control flow.
> Since the caller must be aware of them, generic code generally has to be Result-aware too.
True, but the Rust standard library includes a zero type that can be used to mark a Result-aware function as infallible, making it easy to wrap it with a non-Result type.
That is incredibly domain-dependent. In most general business processing, that information is just noise.
Building a web app? 99.9% of the time, you let exceptions get caught by the http server and return 500 to the client. In a few rare cases where you want to do something else, you catch. If you don't catch, the client still gets 500 - a perfectly acceptable fallback.
Building a GUI app? 99.9% of the time, exceptions in the UI loop should just display an error message to the user in a modal dialog and then get ignored. Sure, you can do something else, but the error dialog is a reasonable fallback.
There is no good reason to torture the whole call stack to accommodate these problem domains.
Functions that never fail are pretty rare compared to ones that do. It's easier to just assume that all functions can fail. Mentally it's a much simpler model.
[1] https://forums.swift.org/t/on-the-proliferation-of-try-and-s...
Code that is error-safe is so rare. Why adopt a pattern that elevates the normal case ("here be errors") to information you have to disclose at every turn?
With "exceptional" errors are are only 2 real recovery options: restart the operation or terminate the operation. Neither of these are typically decided on anywhere where an error might occur in the call stack.
I'm sure the gains aren't tremendous, but lots of C++ design decisions are for slight performance improvements at the cost of less safety. Other examples are unchecked array access by default and unchecked overflow. Whether these are the right decisions or not is debatable, but at least it's consistent.
EDIT: Of course, as the article points out, if you have a high error frequency and many cores, exceptions cause significant performance issues. But in the case where exceptions are _actually_ exceptional, and especially in cases where the only real response to an exception is to log an error and exit, exceptions are exceptionally good (pardon the pun) from a performance perspective.
I agree of course that errors must be handled and logged as appropriate, but the implication that this has to happen by popping from the call stack is not justified at all.
If I'm processing 10 transactions and transaction 8 fails, needs to be retried a couple of times before being skipped for 9, I don't know how you'd do that other than going up the stack to where you're looping over the transaction list.
Because, properly-used, it saves you a lot of effort.
Returning an error type unwinds the stack. If I have a low-level computation that has a number of error cases, and I want the high-level code to intelligently handle some of those error cases and continue processing, that's not doable using return values. Error codes bind the decision of which error-recovery strategy to take (which is only present at a high level) with the details of that strategy (which is only present at a low level).
As a trivial, obviously-fake example - if I have a high-level GUI library that makes use of a low-level division function, I might occasionally divide by zero. Depending on what the GUI library was doing, I might want my divideBy(x, y) function to return 0, 1, the first argument, the second argument, or not return anything because that section of the code will be completely aborted.
Without ambient control flow, if you just have return values, you have to check the return value for every single division operation you perform - and you'll have to re-implement code paths where a division operation failed.
What if you have a long-running operation? If you return an error value when that operation happens, but it turns out the nature of the operation allows you to ignore that error and continue, you'll have to re-start the entire computation. If you hard-code the lower-level logic to ignore errors and continue, then you'll also end up ignoring errors that you really shouldn't.
Error types do not solve these problems - in fact, they require that you duplicate lots of code to, say, make multiple variants of a library that are identical except when it comes to error-handling - or just cause your applications to lose lots of error-handling nuance and bail early on lots of exceptional circumstances that could be recovered from.
This is just as much of a problem with exceptions. "Fix the error and continue" needs some equivalent to resumable conditions, which are generally implemented at the "low level" using coroutines. My understanding is that async-await as a language feature might be able to express these with relative ease, but exceptions alone clearly do not suffice.
If anything, the issue is between checked and unchecked exceptions.
There are `Maybe` and `Either` and they're great at streamlining error handling in pure code, but when it comes to IO most libraries just throw exceptions (including the standard library).
Perhaps the lesson of the last forty years or so is that this convenience wasn't worth adding such a heavyweight feature to the language, but it seems to me that modern languages are still weak at handling overflow.
All the other ones that have born with exceptions don't have this issue, including Ada.
Old school C++ used to have additional try/catch blocks (and performance hit) inserted to ensure that functions match the given exception signature. Also, C++ usually focuses on performance, so all of the bookkeeping required for exceptions could historically be the last 3% or so difference between making your FPS rate or not.
Exceptions are a big issue in Javascript, though mostly because they're absolutely terrible.
Whether to use exceptions or not is a common question in e.g. C#, and several APIs are duplicated to have both exception-based and values-based variants. And Python is oft criticised for using exceptions more than once every blue moon.
― Bjarne Stroustrup
If all memory accesses were done with a function call, e.g. read(void *addr), then by your logic there would be no legitimate need for exceptions.
If you extrapolate into the other direction, exceptions are convenient from a syntactic POV because it avoids littering every operation with an explicit error return mechanism.
A lot of time I see people saying D is unusable (excitedly) for a certain usecase, and you know what if you use the GC a lot it might be, but then I then see them write code that uses raw malloc and free willy nilly. GC can bite you in the ass, but so will basically all memory allocation if you don't understand the real trends of your program i.e. "Nature is a language, can't you read?"
In reality all interactive applications are real-time performance critical. Any normal user would say that an unresponsive application is unacceptable. Sadly the entire stack of our contemporary desktop runtime environments grew out of background batch processing systems. This means that an application becoming unresponsive can be a sign of normal system operation.
Even if you avoid malloc, any memory access can trigger swapping which can block your main thread indeterminately. Or if you are running a high enough amount of CPU-hungry processes concurrently, your main thread can block indeterminately.
Millions of programmers carry on building applications for these systems blissfully unaware of these fundamental flaws. Application runtimes like Electron flourish, riddled with thousands of unbounded operations on every mouse click.
Way too many people believe the difference between malloc/new+free/delete vs a GC is that one is deterministic and the other isn't.
(It doesn't help that textbooks still teach this!)
Both are subject to the whims of the system's memory and, if swap is enabled, IO systems.
And unless you understand your call graph very well, constructing an object that constructs other objects is not something you are likely to be capable of calculating the performance of in a non-GC language.
Same goes for destructors.
The actual difference is that in language without GC's typically have explicit syntax for dynamic memory allocation (though it can happen by surprise in C++ from time to time!), which means if you are writing latency sensitive code, you can just avoid dynamic allocation altogether.
But there is no reason why GC languages can't do the same! In fact, newer GC languages sometimes do, and nowadays C# give you more control over stack allocation, so careful C# coding can also avoid using dynamic memory.
Curious as to the decision criteria.
For example, Eigen is the only library (not in C++, but in the entire programming space) that can optimize your math expressions at compile-time. You can also perform techniques like auto-differentiation by using templates. Meanwhile, Rust still doesn't have full const generics, which are needed to create a math library that's both efficient and easy to use (nalgebra still depends on typenum, which is much uglier than even the most esoteric C++ template stuff you can find!)
See the (unreadable) generated code from Eigen+CppAd Codegen here: https://github.com/google-research/tiny-differentiable-simul...
It's not unique to C++ but yes Eigen is by far and away the most mature option in that space. D templates are significantly better than C++ ones in basically every way, but the work is already done in C++ so I can't be too boastful
There are other as performant languages out there that have a "small" footprint (C, Rust, Zig, etc.), but the combo with the libraries heavily used in robotics, mostly for image and mathematics computation, makes it hard to look away from it: C++ standard library, OpenCV, Eigen, Boost, PCL, ROS.
There may come a day when there is enough momentum and support to look elsewhere, but right now C++ is king in this domain. There is a lot of Python going on too, but usually not for the same type of things.
What else do you suggest?
I'm really heartened in the last few years that FP popularity has caused more people to avoid exceptions and even golang doesn't support them.
Is it unavoidable that that hurts performance badly for the common case? I would think such concurrent changes are rare, so if there’s an asymmetric way to protect against that that’s faster in the happy path exists, that would be a big improvement.
https://en.wikipedia.org/wiki/Readers–writer_lock mentions Read-preferring RW locks, but (from cursory reading) doesn’t say how much that can help.
RW lock is an option that helps but involves breaking the ABI of EVERY shared library.
I do agree with the mentioned design issues though.
This introduces a whole bunch of design issues, but isn't `panic` (in unwinding mode) basically C++'s exceptions, implementation-wise? Couldn't we use `panic` + `catch_unwind` as a poor man's exception system, should the performance situation really require so?
Also, could we maybe add an attribute `#[exceptional]` to enum variants in a match, that would result in a match implementation closer to exceptions, implementation wise?
Further, as you can see in other comments, it's not an uncommon belief that exceptions simply shouldn't be used for A (and thus it's reasonable that they aren't optimized for something they aren't supposed to be used for).
I am convinced that maintaining ABI is a huge technical debt to C++. And that is largely occasioned by shared libraries.
Rust IMO made a fantastic decision to emphasize source distribution (and making it super easy via cargo and crates).
It does seem like at least in theory it should be possible to create a non-locking / blocking exception unwinder, if there is no actual contention between the threads. If that can be done then it seems like the solution should be to do that rather than abandon a whole language feature. This is a bit like the Python GIL question. I would say if the language spec means you have to have a "GIL" in any context in a high performance language like C++ then it ought to be addressed at the spec level.
> The second problem could potentially be fixed by a sophisticated implementation, but that would definitively be an ABI break and it would require careful coordination of all components involved, including shared libraries.
This seems like a perfect application for an rwlock. Threads that throw acquire for reading, so can occur in parallel. Threads that load or unload shared libraries (a pretty rare occurrence outside of process start) acquire for writing and therefore block reads from the table from throwing threads. The shared library loader shouldn't throw during this piece.
Come to think of it, throwing should also be pretty rare. C++ exceptions are not really about common control flow, but truly exceptional circumstances. So the high cost serializing parallel throwers doesn't even sound like that huge of a deal. But as I've said, it seems pretty easy to improve upon it.
[Edit: It would seem the article does say something about how an rwlock is not feasible with the current implementation ... It doesn't sound terribly convincing]
Edit: nope. This idea is discussed later in the paper (not fully ruled out but the answer may still require ABI changes for more subtle reasons)
I find the paper’s argument thin on why this could only be done in an ABI incompatible way.
But look at some of the costs we have to go to great lengths to get around. std::sqrt has to check for invalid inputs and that means a branch, often even when the code is checked prior or an ASSUME( f > 0.0 ) type thing is expressed. This inhibits auto-vectorization. On some compilers ASSUME( not std::isless( f, 0.0 ) ) can tell the compile that the number is real and >= 0 which elides the branch. But the math-error/errno issue is a big hinderance too.
A lot of the costs is the branch itself and telling the compiler it is not needed can boost hot path perf more than the rare cold paths.
Just throw.
But returning null is an obvious choice.
Combined types which return the structure or an error are another. Has the advantage of returning error information, why failure.
There are many ways
It's not because C++ is not an everything-is-a-reference language.
> Combined types which return the structure or an error are another. Has the advantage of returning error information, why failure.
Sum types are great, but changing constructors to return sum types would break all existing code. It would also lead to a whole slew of questions about how the construction of arrays of values would work. (Does an array of an objects now turn into an array of wrapper types? Byebye SIMD optimization!) It would also break the consistency of constructing POD vs non POD types, which would make writing templates that need to generalize over both a huge PITA.
In the gaming case(no rtti, no smart pointer, no exceptions(thus meaning ctor|dtor are "unsafe")), what else do you leave with c++ then?
I use c++ but I'm always struggling with yes-or-no for exceptions.
c++ is deeply rooted with exceptions, bad or good.
C++ without exceptions is great. Google doesn't use exceptions and they still use STL. Gamedevs usually don't use exceptions and they're fine. Just use some other mechanism for errors, like error codes, absl::Status, std::expected, etc.
Most STL operations and iteration etc will throw exception for errors, if you disable exception, any of those errors, be it recoverable or not, will just std::terminate, which might not be ideal.
OK, so replace the mutex with a reader writer lock. Shared library loading is incredibly rare, and dear god I hope nobody seriously does it and expects it to work during unwinding.
Legacy projects (I am not being pejorative, they are important) which must be maintained aside, should not C++ be deprecated?
We have many new languages, we always had C. What does C++ give us in 2022 that makes up for the enormous cognitive load of understanding and keeping up with it.
That said while I used to use C++ for nearly everything, these days I use Elixir for app dev whenever I can, and Ruby and bash for scripts. However if I were going to write a desktop app today I'd most likely go for C++ so I could use Qt. I do really need to try out new GTK though, sounds like it's gotten really great.
As for what do you get? Well, compared to C you can get higher programmer productivity and compared to any language other than C, you get great performance.
The short answer is probably that (unsurprisingly) C still does not scratch the particular itches that C++ was created for, and non of the new languages hit all of them well either. Plus network effect.
- I've been using C/C++ since my first year of undergrad so I'm already familiar/comfortable with it
- Java has too much boilerplate (my mind is still stuck in Java 8 so maybe things have changed since then)
- I found myself spending more time fighting the Rust/Go compiler than actually solving my problems
- I don't want to package a web browser for a simple 1-2 UI application
If I were starting a greenfield project today, I might choose C++ for it, depending on what it was. (Pick the best tool for the job, and all that.) But if I picked C++, I would not use the full enormity of the entire C++ language specification. I would use the parts that helped with the program I was trying to write, and explicitly not use the rest of it.
This is basically the path Rust has chosen. I’m curious if it’s _actually_ too slow for C++. I feel like the answer must be no.
You cannot draw useful inferences from those cases.
> For fib we see a slowdown of approx. 60% compared to traditional exceptions, which is still problematic.
C++ genuinely traded “error” cases (made them even costlier) so that the happy path of exceptions would be cheaper. I don’t remember whether that’s the case in C++, but in some language/implementations (used to be a big issue in V8) just having a try/except would drastically deoptimise a function, even if the exception never happened.
Sometimes that happens rarely but blocks many threads is a huge pain to deal with.
No.
We disable exceptions in video games for dumb historical reasons that no longer apply.
There are tons of games where exceptions are perfectly fine. Though still more often used for "probably going to crash soon" situations than otherwise, I'd wager.
Still prefer them to the average python error message.
Consider this crap small code fragment. Any function that returns void is almost certainly wrong.
That said, exceptions should have been allocated on the stack, and catch handlers should have been closures that get called with the exception object. I guess this can't be implemented now.
As for multi-threading unwinding, the issue there is that unwinding tables are part of the shared objects whence the associated object code comes, and it often has to be possible to unload loaded shared objects, so now what? The situation shouldn't be bleak though: a shared object should never be unloaded while there are threads executing its code, so it should be possible to arrange a slow unload-time operation to update the the process-wide unwinding tables -- think of user-land RCU if you like.
The exception handling code should not have to synchronize around unwinding, except that when unwinding completes it might have to notice that there's a pending unload, so wake the thread that is waiting for unwinding. Or maybe not even, because maybe unloading could do something truly breathtaking like check that no thread's stack includes return addresses from the object to be unloaded, and then unwinders would never need to step into the unwinding tables from that object.