Hacker News Reader

> The idea is that optionals allow you to pass optional data by value.

Yes, and kj::Maybe was doing the same before std::optional was standardized.

It's disappointing that the committee chose only to solve this problem while not also solving the problem of forgetting to check for null -- often called "the billion-dollar mistake".

> Dereferencing null optionals is UB for consistency with dereferencing pointers. All uses of operator* should have the same semantics

My argument is that `std::optional` should not have an operator* at all. `kj::Maybe` does not have operator* nor `operator->`.

> If you want to dereference an optional that may be null, use the .value_or() method. For the times when you absolutely know the optional has a value use operator*.

This is putting a lot of cognitive load on the developer. They must remember which of their variables are optionals, in order to remember whether they need to check for nullness. The fact that they use the same syntax to dereference makes it very easy to get wrong. This is especilaly true in modern IDEs where developers may be relying on auto-complete. If I don't remember the type of `foo`, I'm likely to write `foo->` and look at the list of auto-complete options, then choose one, without ever realizing that `foo` is an optional that needs to be checked for null.

In KJ, you MUST write:

    KJ_IF_MAYBE(value, maybeValue) {
      use(value);
    } else {
      handleNull();
    }

Or if you're really sure the maybe is non-null, you can write:

    use(KJ_ASSERT_NONNULL(maybeValue));

This does a runtime check and throws an exception if the value is null. But more importantly, it makes it really clear to both the writer and the reader that as assumption is being made.

> My argument is that `std::optional` should not have an operator* at all. `kj::Maybe` does not have operator* nor `operator->`.

That’s fair.

>> If you want to dereference an optional that may be null, use the .value_or() method. For the times when you absolutely know the optional has a value use operator

> This is putting a lot of cognitive load on the developer. They must remember which of their variables are optionals,

Not really. Teams with programmers that are bad at keeping track of the state of their variables can simply have a policy to always use .value_or()/.value()

C++ doesn’t impose this on its users because it generally assumes its users are responsible enough to make their own policy.

> The fact that they use the same syntax to dereference makes it very easy to get wrong.

I disagree, the operator* has the same semantics as pointers did, making it no more semantically hazardous. There exists other methods on optional that have the behavior you want.

> always use .value_or()/.value()

But neither of these solve the problem either. Neither one forces the programmer to really confront the possibility of nullness and write the appropriate if/else block. Throwing an exception rather than crashing is only a slight improvement IMO.

> I disagree, the operator* has the same semantics as pointers did, making it no more semantically hazardous.

It was already severely hazardous with pointers, that's the problem.

>> always use .value_or()/.value()

> But neither of these solve the problem either. Neither one forces the programmer to really confront the possibility of nullness and write the appropriate if/else block.

.value_or() actually does and you can certainly add a lint check against dereferencing optional or using .value() if you’d like. C++ does not Yet provide Case-style syntax for handling variants like rust, outside of macros and the standard library will certainly not define macros.

I think what you have done for your codebase makes sense based on your preferences but I think the standard optional works pretty well given the variety of code based and styles it’s intended to support.

> It was already severely hazardous with pointers, that's the problem.

So then don’t use the dereference operator.

kj::Maybe has an `orDefault()` method that is like `.value_or()` but I find that it is almost never useful. You almost always want to execute different logic in the null case, rather than treat it as some default value.

Then you can make a quick helper subroutine that adds a monadic interface to optional and you can lint away all non-conforming uses of optional.

Both problem could have been solve looooooong ago by introducing a type modifier akin to const that carries if a value is verified (or safe or non-null or other. Pick your synonym).

   int * p; // maybe null!
   int * verified p; // guaranteed non-null!

A looooong time ago (circa... 1994-1995) I designed a hierarchy of smart pointers and had a variety for non-null so that you could declare a function like:

   void foo(non_null_ptr<T>& p);

And know that you don't have to verify for null. All enforced at compile-time. (via the a function on ptr<T> returning a non_null_ptr<T>).

With language support around if() and others, C++ could have mde it even more convenient. Even C could have introduced such a tyupe modifier. Whenever I read about pointers being unsafe and how optionals and maybes are the solution, I roll my eyes, because non-null-ptr do the exact same thing.

The funny thing is C++ has a non-null ptr (with no language support guarantee though): references. Unfortunately, the language made them not resettable, which makes them unusable in many scenario when you'd want them to change value over time, like in most classes members.

Isn’t “non null optional” the same as just passing the base type?

By reference? Yes.

But the idea of a verified type can be extended by using the verified modifier on your own type. For example, you could have a verified matrix type, where the matrix is guaranteed to be valid, non-degenerate. You can apply it to:

   - matrix
   - vector
   - input data of any sort

And if teh compiler allowed the programmer to declare their own type modifier, the world is your oyster: you could for example tag that a matrix is a world matrix while another a local matrix and provide a function that converts from one to the other...

I wrote a small blog post about the idea:

https://www.spiria.com/en/blog/desktop-software/hypothetical...

Are you aware of std::reference_wrapper? https://en.cppreference.com/w/cpp/utility/functional/referen...

> It's disappointing that the committee chose only to solve this problem while not also solving the problem of forgetting to check for null -- often called "the billion-dollar mistake".

it's likely ~30 loc to wrap std::optional in your own type that checks for null. if std::optional checked for null and these `if` branch showed up as taking the couple nanoseconds that make you go past your time budget in your real-time system (especially if people were doing things like if(b) { foo(b); bar(b); baz(*b); }) then you have to reimplement the whole of it instead.

Don't forget that you can still use C++ on 8mhz microcontrollers.

The ideal situation is then to not use std::optional for those cases rather than to make std::optional next to useless for it's stated case.

If it gets in the way of your goal on an 8Mhz controller, take the optionality check out of your tight loop and convert to a null pointer safely where it doesn't matter.

Deeply embedded is already used to picking and choosing features, or explicitly running with the bumper rails off in the subset of cases where it matters. We like the normal primitives not being neutered for us because we still use a lot of them outside of our tight loops.

> than to make std::optional next to useless for it's stated case.

This is such a ridiculous and obviously false assertion that it’s indistinguishable from satire. Optional is widely used and was modeled from a pre-existing boost class which was itself widely used. Do you actually write C++ professionally?

Yes.

And I think (hope?) that std::optional is going to make it's way into the dust bins of history like auto_ptr.

When the answer is "just don't deference it if it's null", then why not just use a pointer in the first place?

Again, optional is different from pointer because it offers value semantics.

I know they're different (and the construct unfortunately named optional has some occasional uses); it's just that the semantics of optional don't help it be used as a classic optional type.

> it's just that the semantics of optional don't help it be used as a classic optional type.

what do you mean "classic optional type"? boost.optional has worked like that for something like 20 years - it's been in C++ for longer than Maybe has been in Haskell.

It's been in Standard ML since the 80s.

> it's been in C++ for longer than Maybe has been in Haskell.

Tangentially, how did you conclude that? Haskell has around since 1990 but boost only since 1999, as far as I can tell.

https://en.wikipedia.org/wiki/Haskell_(programming_language)

https://en.wikipedia.org/wiki/Boost_(C%2B%2B_libraries)

Again, I'm not arguing that operator* should check for nullness, I'm arguing that it shouldn't exist.

With `kj::Maybe` and `KJ_IF_MAYBE`, using the syntax I demonstrated above, you check for nullness once and as a result of the check you get a direct pointer to the underlying value (if it is non-null), which you then use for subsequent access, so you don't end up repeating the check. So, you get the best of both worlds.

> it's likely ~30 loc to wrap std::optional

It's even easier to replace std::optional rather than wrap it. The value of std::optional is that it's a standard that one would hope would be used across libraries. But because it's flawed, people like me end up writing their own instead.

> So, you get the best of both worlds.

I would really not call code that looks like this "best of both worlds"

    KJ_IF_MAYBE(value, maybeValue) {
      use(value);
    } else {
      handleNull();
    }

when compared to

    if(!value) 
      handleNull();
    use(*value);

Kind of unfair that you didn't include the `else` or the braces in your version, just to make it look shorter.

> Dereferencing null optionals is UB for consistency with dereferencing pointers

The point of optional is to avoid being consistent with the bad parts of pointers. And making it undefined rather than a guaranteed crash is even crazier.

Ford doesn't sell cars that burst into flames for consistency with the Pinto.

And usage of the dereference operator isn’t intended for uses that would cause things to burst into flames. If you don’t know the state of your variables or you don’t trust your coworkers to know the state of their variables, you can enforce the use of value_or() in your own projects. You don’t get to force superfluous branch stalls on the general C++ user base.

I think your replies in this thread show a complete misunderstanding of what std::optional is for (or at least, what it should be for, in my opinion).

std::optional is for modeling a value that may be null. If the value may be null then you must check if it is null before you dereference it. There is no "forcing of branch stalls", because if used correctly (and designed correctly, which std::optional is not, sadly) it is merely a way for the programmer to use the type system to enforce the use of null checks that are necessary for the correctness of the program anyway.

If you and your coworkers find yourself in a situation in which you know that the value of a pointer cannot be null, then you should not model that value with an optional type, and then you will not only not be required to add a null check, it will be immediately obvious that you don't need one.

> If you and your coworkers find yourself in a situation in which you know that the value of a pointer cannot be null, then you should not model that value with an optional type, and then you will not only not be required to add a null check, it will be immediately obvious that you don't need one.

Hmm I think you’re suffering from a lack of imagination and real world experience with efficiently storing data in C++.

There are certainly cases where it makes the most sense to instantiate your value within an optional wrapper while at the same time there being instances within your codebase where that location is known to be non-null. I’m surprised I even have to say that.

An obvious case is when using optional as a global. Other cases are when you’ve checked the optional once at the top of a basic block to be used multiple times after.

> An obvious case is when using optional as a global.

Well, ok, although I think we were doing fine storing such values in unique_ptr. Now you're going to come back and say that you can't ever afford a double indirection when accessing globals, and if so, fine. But you still could have very easily written your own wrapper that suits your needs without demanding that std::optional be relaxed to the point where it cannot provide compile-time safety guarantees.

> Other cases are when you’ve checked the optional once at the top of a basic block to be used multiple times after.

Disagree. The way optional types are supposed to work (and the way I have used them in real code) is that you check it once, and in doing so, you obtain a reference to the stored value (assuming it the check passes). Further accesses to the value thus do not require checks. The type system is thus used to model whether the check has been done or not, and helps you write code that does the minimal number of checks required.

You seem to think everyone else in this thread is an idiot, but I promise you I have written real code with very strict optional types (similar to kj::Maybe) without introducing unnecessary branches.

optional has a bunch of reason's why it is better than a pointer too(it cannot be incremented, it's not a failure to initialize, it doesn't implicitly convert to things readily...). Unfortunately we don't have a monadic optional or an optional with a reference member. Both would be very useful. Value or suffers from having to evaluate the or part in all cases, but if we had .transform/.and_then/.or_else members would be really nice. Optional of a reference would allow for a whole swath of try_ methods instead of the idiomatic at( ) interface for checking bounds and retrieving in one call. at( ) suffers that it forces the choice of an exception or checking the index for being in range outside the call and then operator[] is what you want.

Some monadic method would be nice but as you probably know, it’s trivial to implement one yourself.

You can store a reference in optional using reference_wrapper https://en.cppreference.com/w/cpp/utility/functional/referen...

reference wrapper breaks in generic contexts.

Could you provide an example?

You have to add special handling for the case that T == std::reference_wrapper<T> so you can call .get() on it to expose the underlying value. In the case of std::optional vs a pointer type (raw or smart) you can consistently use operator* to get to the underlying value. I think this is what was meant.

also if you are doing something like decltype( *val ) to get at the underlying type.

well, representing an optional as a pointer, while nice syntactic sugar, is itself a design decision and adding UB there is not nice.

In practice I believe that all standard libraries have a lightweight check mode that can assert even in release mode.