Optional Value Handling in Nim | Hacker News Reader

Optional Value Handling in Nim

peterme.net

43 pointsmikenew24 comments

One of the things I don't like about Option[T] in Nim is when you are handed an optional large structure as a function argument, such as "Option[seq[Thing]]" or "Option[BigObject]", or those types as fields of an object passed as argument.

It works fine, but the act of unwrapping the value inside with x.get copies the large data structure inside to return it. So it's incredibly slow to access the value inside. Even one such access can be unacceptably slow.

Receiving the optional itself as a function argument is fast, because Nim passes a pointer for those arguments. The problem is when you need to use the value wrapped inside.

Nim encourages "value oriented programming", where things are written as value types, on the assumption that most of the time Nim is able to use temporary read-only aliases to data instead of copies. So this is the style of passing optionals around that I see in typical code.

If you have been handed a mutable option ("var Option[..]") that's different. Then you can unwrap the value inside as a mutable as well, which avoids the copy.

But you don't tend to be handed a mutable option in a function argument, except from code that was written in an unusual style with performance in mind, and a contract outside the type system: "I'm giving you a var, but only for performance reasons; don't write to it!".

See the lent annotation? No copying involved. Its a hidden pointer.

https://nim-lang.github.io/Nim/options.html#get%2COption%5BT...

It's a great idea. I really like a lot of Nim's language design, including from a performance perspectice.

If only things were implemented as reliably and nicely as the design suggests. I think people like the neat documented language and take for granted that the implementation must have all things that are implied or suggested by the design, when it's not quite up to natural expectations in practice:

> @mratsim: As a potential impact, I tried to use options in my RayTracer, but they were 5x slower (not just a couple dozen percents like the iterator issue in #14421) than passing mutable parameter + bool

> @araq: Well nobody claimed Option[T] is a lightweight abstraction.

> @mratsim: I was misled by Rust code :/

https://github.com/nim-lang/Nim/issues/14420#issuecomment-63...

Without going into details of any particular problem, I can safely say that when I profile Nim code at runtime, I see a lot more copying than I think was intended by authors.

And when I look at the generated C code, I see a lot of temporary objects and redundant copies into and out of them, where it's obvious just from reading a few lines in the C code they are redundant. Not just copies that could be cleverly turned into pointers by lent etc., but temporaries that trivially don't need to be there even without fancy lent/borrowing optimisations.

> And when I look at the generated C code, I see a lot of temporary objects

yea it sucks, hopefully it would be fixed soon.

https://github.com/nim-lang/Nim/issues/18341

The comparison of Nim to Ada seems appropriate. I've never used Ada but wrote a paper on it in school and found the concepts intruiging. Pretty interesting that a design by committee language had such rich integer types.

Nim looks like Python but really it reminds me more of a nicer looking Pascal. It's been great having ranged integers in Nim, especially when doing embedded work, without having the verbosity of Ada.

Simple Pascal is capable of having different integer types that do not mix, or have controlled ranges, etc. To compare with Ada, I'd like to see Controlled and Limited types :)

Ah I didn’t recal ranged types in pascal. Granted my pascal experience was back in high school.

I’m not too familiar with controlled or limited types, but it seems the primary goal is control of resources. Nim’s compiler includes copy, sink, and destroy procedures for types [1]. The new ARC GC is deterministic, so you can ensure that once an object leaves the current scope it’s destructor will be invoked. It’s handy to cleanup C structs that are manually allocated by wrapping them in a Nim type. Makes C apis so much safer to use with minimal extra overhead.

1: https://nim-lang.org/docs/destructors.html

The key idea: Optional[T] is more ergonomic through pattern matching, because inside the matched case, we definitely know whether it's carrying a value or not. The whole problem of nullability gets an elegant solution.

The fun premise from the beginning is that booleans do not need their special type, they are effectively Optional[Void], or for FP-heads here, Optional[Unit]. That is, it just does not need to carry a value. But when an Optional does carry a value, a lot of if-then-else logic for handling that value just gets eliminated, replaced by a pattern match that combines both the boolean aspect and the value-handling aspect.

> The fun premise from the beginning is that booleans do not need their special type, they are effectively Optional[Void], or for FP-heads here, Optional[Unit]. That is, it just does not need to carry a value. But when an Optional does carry a value, a lot of if-then-else logic for handling that value just gets eliminated, replaced by a pattern match that combines both the boolean aspect and the value-handling aspect.

We could take it step further.

Option[T] is just Result[T, Unit]

Then a boolean is Result[Unit, Unit]

Option types require do-notation / computation expressions to be practical in my view. Otherwise you get the pyramid of doom!

    match x with
    | Some y -> 
      match y.Foo with
      | Some z -> 
        match z.Bar with
        | Some w -> printfn $"{w.Qux}"
        | None -> ()
      | None -> ()
    | None -> ()

Compared to...

    option {
      let! y = x
      let! z = y.Foo
      let! w = z.Bar

      printfn $"{w.Qux}"
    }

Funny quirk in Java, the compiler checks that calls to `opt.get()` occur inside an `opt.isPresent()` branch!

    if (opt.isPresent()) {
      println(opt.get());
    }

The pyramid of doom means that you're explicit about error handling at each step though, which doesn't look easy with the do notation.

You must understand the semantics of the monad you are operating in, but both cases are still handled in the do-notation version.

> booleans do not need their special type, they are effectively Optional[Void]

The fact that two types are structurally isomorphic doesn’t mean that it’s necessarily a good idea to make them the same type.

I think the actual observation of the article is that many booleans are guards for the presence (or validity) of a value, and that modelling those as Optionals binds the result of the guard and the applicable value in a way which can prevent later mistakes.

It's not arguing at all booleans should be treated this way.

That makes more sense, although I can’t say that "many booleans are guards for the presence (or validity) of a value" matches my experience.

Many of those booleans aren't necessarily named.

Any "is(n't) null" check on a nullable includes a boolean expression. That's quite a few booleans right there.

Using an optional (and accessing it with either fmap or a match expression) removes the value from the scope of the "wasn't present" branch, eliminating a possible source of mistakes.

Using an optional with an isPresent test does not. And you see that boolean pop up again in the return of that method. I think that's what they're getting at.

This reads a lot like Rust does it's error-handling too.

Right. I think Rust was the first to propose “?” operator as a syntactic sugar to Result and Option monads. (i am not 100% though)

ref: https://m4rw3r.github.io/rust-questionmark-operator

Ruby got the safe navigation operator (&. to safely call methods on a nil-able value) in 2015, with Ruby 2.3, and credits "C#, Groovy and Swift" for inspiration there. So while it isn't exactly "?" (I think Groovy does use the ?), the semantics are close enough IMO.

Ref: https://www.ruby-lang.org/en/news/2015/12/25/ruby-2-3-0-rele...

Yes, Groovy has used '?' in various forms for some years now as well:

https://www.logicbig.com/tutorials/misc/groovy/safe-navigati...

https://www.logicbig.com/tutorials/misc/groovy/elvis-operato...

I think Kotlin may have had it before 2015.

C# added this operator in 2014 with C# 6 ( and the simultaneous release of VS2015.)

https://docs.microsoft.com/en-us/dotnet/csharp/whats-new/csh...

https://docs.microsoft.com/en-us/archive/msdn-magazine/2014/...

Roslyn compiler, also supporting this operator, was released earlier in the same year, too.

C#'s Null-Conditional operator isn't the same as Rust's Try operator. The Try operator will early return from a function, while the Null-Conditional operator conditionally executes the rest of the expression if the receiver isn't null, or else evaluates to null. The equivalent in Rust is the `and_then` method.

C# also has a Null-Coalescing operator (`??`) which behaves like Rust's `unwrap_or` method.

<tangent> Article content quality aside, I found the layout absolutely delightful.

Reply on news.ycombinator.com