Prologue: Full-Stack Web Framework Written in Nim | Hacker News Reader

Prologue: Full-Stack Web Framework Written in Nim

github.com

166 pointselcritch55 comments

I've been writing an interpreter with Nim and overall the language is really good.

One feature that I'm excited for is DrNim[1]. It's not distributed with the stable version just yet (you have to compile it yourself, but that's very straightforward), but it's supposed to bring refinement types to Nim. These allow you to write things like:

    proc safeDivide(a: int, b: int): float {.requires b > 0.} =
        a / b

and DrNim will check at compile-time that all code paths lead to `b` being greater than 0, e.g.:

    var x = stdin.readLine.parseInt
    safeDivide(1, x) # DrNim will error at compile-time 
    var y = stdin.readLine.parseInt
    if y > 20:
       # no error since DrNim (using Z3) can
       # prove that y is always greater than 0 here
       safeDivide(1, y)

[1]: https://nim-lang.org/docs/drnim.html

I didn't know that this is called a refinement type! I've been wanting it badly, both in Nim and in Rust. Now I have a way to search for what I'm looking for to learn more.

Would it be possible to use this to ensure a string matches a certain pattern? For example, maybe I want the string to look like a phone number or zip code. I know I can use an abstraction to do this and have some level of safety, even without a statically typed language, but I love the idea of implementing that with primitives. As far as I know it isn't really a thing though. I'm assuming a refinement is only really a refinement of a type within the same system, and doesn't allow for special logic like pattern matching. That would be amazing though.

I've spent so much of my career working with dynamic languages, it's hard to know exactly which tools are available to me with these kinds of type systems. I love it though. Learning to leverage types has been such a fun shift.

I don't think you can use _refinement types_ for this, at least not as of now (e.g. you can't do {.requires validEmail(email).}), but you can use distinct types[1] in nim to model what you want:

    type Email = distinct string

Now Email is a type that is incompatible with string, even though its runtime representation is a string (i.e. no additional overhead). To convert a string to an Email, you can call Email("a"). In real-world code, you would call that constructor only in a few "safe" functions:

   # I'm raising an exception, but you could use an option or a result type
   proc validateEmail(str: string): Email =
      if email.contains("@"):
          return Email(str)
      raise newException(InvalidEmail, "not valid")

Then, in your client code you can have:

    proc createUser(email: Email, password: Password) =
       ...

And the type system guarantees that you only call createUser with "Email"s, not plain strings. Of course, you have to be dilligent not to use the Email constructor directly and instead use the validateEmail function to create "Email"s.

[1]: https://nim-lang.org/docs/manual.html#types-distinct-type

EDIT: The sibling comment mentioned io-ts.

The pattern for doing this in io-ts (and TypeScript in general) is very similar to Nim, although they call it "branding"[2] as it abuses the structural type system to simulate nominal typing.

[2]: https://michalzalecki.com/nominal-typing-in-typescript/#appr...

Not sure about Nim, but you can do this in TypeScript. There's even a library called io-ts that allows you to create composable types of this form for runtime validation which then turns the result into proper types.

Wow, that's really helpful. I'm working on a TS project at the moment and that might come in handy. Thanks!

Seems like subset of dependent types to me

For me that's a game changer in terms of using Nim for future projects. I've used LiquidHaskell[1] a bit but always been hesitant to use in production just because I'm not confident enough with the Haskell type system. Nim's been on my radar for a while but have held off diving into it for whatever reason, so this is very cool to see.

[1]: https://ucsd-progsys.github.io/liquidhaskell-blog/

This looks really nice. Do you know if type narrowing is a form of this as well? TypeScript has this pretty neat feature where if you have a variable x of union type A | B and you're in a code path where you've checked if x is of type A then the type is automatically narrowed to A in that code path without you doing any casting etc.

    function num(x: number | Array<string>): number {
      if (typeof (x) == 'number') {
        return x + 10;
      } else {
        return x.length;
      }
    }
    console.log(num(5)) // 15
    console.log(num(["A", "B", "C"])) // 3

I think they just refer to it as type narrowing but it feels kind of similar in concept, so I'm wondering if it's a limited form of refinement types.

At first it looks like a guard, but if it's a compile-time guard, it's even better then the original concept!

Reminds me of static analysis tools like sonarqube, but even more integrated

Looks more like flow typing than refinement types to me.

The (static) type of 'y' changes seemingly implicitly in the body of the 'if'. It's an 'int' at assignment but needs to be an 'int {.requires ??? > 0.}' (or however the one would write that type literal in Nim), and some flow typing determines than that that cast form 'int' to 'int {.requires ??? > 0.}' is safe in the body of the 'if', and performs this cast implicitly.

The question now is: Is 'int {.requires ??? > 0.}' a type on its own in Nim? It doesn't seem like that as 'b' has 'int' ascribed as its type, and not 'int {.requires ??? > 0.}'.

It's also not dependent typing as the constraint on 'b' can't be an arbitrary function. (I guess, please correct me if I'm wrong; but in that case this typing wouldn't be decidable, even with the help of SMT).

The type of 'safeDivide' could be a "depended function type", tough.

> Looks more like flow typing than refinement types to me.

I'm not super familiar with the theory behind refinement types and I could be wrong :)

My familiarity with refinement types comes from things like Liquid Haskell and F*. If you look at Liquid Haskell's manual[1], their definition of refinement types seems to pretty much match what DrNim is offering here.

I've never seen the term flow typing used outside of TypeScript-style analysis, where that means narrowing types like `string | null` to `string` inside an `if (x != null)` block.

> The (static) type of 'y' changes seemingly implicitly in the body of the 'if'. It's an 'int' at assignment but needs to be an 'int {.requires ??? > 0.}' (or however the one would write that type literal in Nim), and some flow typing determines than that that cast form 'int' to 'int {.requires ??? > 0.}' is safe in the body of the 'if', and performs this cast implicitly.

I think the hard part is that the language must keep track of these implicit types (e.g. ranges of values an integer can be) inter-procedurally, i.e. throughout the whole call graph.

But yes, the type does change implicitly, and I don't think `int {.requires ? > 0.}` can be expressed as a type in Nim. But I was also under the impression that you can't express that "explicitly" in e.g. Liquid Haskell (I could be completely wrong).

> It's also not dependent typing as the constraint on 'b' can't be an arbitrary function. (I guess, please correct me if I'm wrong; but in that case this typing wouldn't be decidable, even with the help of SMT).

I think you're right. From my understanding, dependent types are more powerful than refinement types but also more cumbersome to use, refinement types can use SMT solvers and are generally more approachable IMO. There's a nice discussion here[2].

[1]; https://ucsd-progsys.github.io/liquidhaskell-tutorial/01-int... (see section 1.2) [2]: https://www.reddit.com/r/dependent_types/comments/ay7d86/wha...

This looks amazing, off to read about it now! Thank you!!!

I'm curious: Is there anything you're using for parsing in Nim?

I'm in the process of rewriting the language from typescript to nim, so right now it's a recursive descent parser written in typescript[1]. Typescript generates the textual bytecode and that feeds into a Nim interpreter.[2]

I think I'll keep the recursive descent parser even after rewriting to Nim. It's easy to change the grammar and understand how it works, and I can have good error messages & handling.

IMO the main problem is that right now the code for parsing infix expressions is fairly repetitive, I'd like to eventually use Pratt parsing for that.

[1]: https://github.com/frankpf/kiwi/blob/vm/src/parser.ts [2]: https://github.com/frankpf/kiwi/blob/vm/src2/src/interpreter...

I played with Nim this weekend.

It's a shame that it did so many things right yet went with unqualified imports by default.

After being spoiled by other languages, something even Node.js got very right, I just don't want to try to get work done anymore in a language where you go `import Lib` and then a bunch of unqualified identifiers like `foo()` are available in the code.

I liked that Ruby feature as a beginner. But it didn't take long for me to realize that knowing where things come from is worth the 500ms one-time-cost you spend qualifying each import.

I code Nim for fun some weekends and agree that totally sucks, and what's worse it seems like it's idiomatic to write Nim that way.

However, for us we can do:

    from "./crawlers/leetx.nim" import nil

    leetx.fetchLatest()

That forces you to use the qualified version and doesn't contaminate your space.

That always bothered me too but never saw any good justification for that. I feel like it's just a straight up flaw in the language, unless someone can tell me why you would ever want unqualified imports.

See https://news.ycombinator.com/item?id=23452857

In short: unqualified imports don't play well with operator overloading, templates, or generics.

I think you mean "qualified imports don't play well" from reading your other comment.

Woops, yes, you're correct

It hasn't bothered me with types and auto-complete combined with unified-call syntax (UCS) after a bit of use. Not sure if UCS and qualified imports would work so well. You can import specific names (similar to Python) like `import posix as ps`. Is that what you mean by qualified imports?

Nim supports both qualified and unqualified imports[0]. One uses `from module_name import nil`. Then every access to members of that module must be qualified.

From my viewpoint (I'm sure there are others) I see two main reasons why unqualified imports are the default: operators and templates/generics.

First, operators[1]. In Nim, one can implement an operator for any type, without extending that type, and without even implementing the operator in the same module as the type definition. If qualified imports were the default, then

  a + b

would by default be

  module_name.`+`(a, b)

(The backticks are used to force the plus sign to be recognized as an identifier)

Yes, one could hypothetically perform an "unqualified import" for that specific symbol, but what if you have 5+ operators to import (like for a math library)? It would get tedious rather quickly.

Now, templates. If you're not familiar with Nim, templates are similar to C templates, except that rather than a template performing textual substitution, it performs syntax tree substitution. As an example, this is how "less than" is implemented:

  template `>`*(x, y: untyped): untyped =
    ## "is greater" operator. This is the same as ``y < x``.
    y < x

If qualified imports were the default, templates would be almost useless. Something like this:

  template toJsonString(value: untyped): untyped =
    $(toJson(value))

where `$` turns its argument into a string, and toJson turns its argument into a JSON node type, would fail because the template doesn't qualify any of those procedures.

The same reasoning for templates also applies to generics (and might be a bit more familiar). A generic doesn't know where its input type comes from ahead of time, so it can't know how to qualify function calls.

Python, C#, and other class based languages deal with these complications by using classes as a kind of "automatic unqualified import" signal. Though it doesn't have classes, Go uses a similar mechanism (explicit methods types that must be defined in the same package as the type implementation). Since Nim has neither method types nor classes, it just uses unqualified imports.

I agree that unqualified imports are a pain for code navigation, but with the right tooling that navigation can be made easier (I know there's a Nim plugin for VSCode, and I believe there's ones for Vim and Emacs?)

[0] https://nim-lang.org/docs/manual.html#modules-from-import-st...

[1] https://nim-lang.org/docs/manual.html#syntax

[2] https://nim-lang.org/docs/manual.html#templates

Operators are an argument for why it should be opt-in. Most code you write doesn't use imported operators, and when you do use them, you aren't using many of them. Encumbering everyone's code for code that they might write isn't usually the right trade-off.

Just make provide the feature via opt-in. Other languages do this just fine:

    from Lib import (foo, (+)) // Just +
    from Lib import (foo, (...)) // All operators

Good defaults are important so that the ecosystem develops good conventions. Else you get "well, you can qualify imports if you want" as seen in this thread. After reading code all weekend, nobody qualifies imports in Nim.

And I also believe in escape latches, not absolutism. If your templating argument were the best reasoning, then it makes sense for that to be an isolated idiosyncrasy rather than a reason for all code to pay the penalty of unqualified imports by default.

Just like you can opt-in to importing entire namespaces into scope in other languages, but you only do it on occasion when it makes sense, not by default.

Anyways, I know this is classic HN bikeshedding, but Nim gets mentioned a lot on HN, the developer clearly has a good eye for attention to detail and thoughtfulness. So it surprised me that Nim had what I consider one of the fundamental mistakes in programming languages along with, say, a lack of first class Optional<T>.

What is the perceived penalty of unqualified imports in Nim specifically? The only one that seems valid is you can't immediately see the module a proc is defined in, but you can opt in as you say with module qualifiers if you want to.

None of the "namespace pollution" stuff applies to Nim since ambiguity is a compile time error and there's no performance penalty for having everything imported as Nim uses dead path elimination.

It's true that custom operators aren't implemented very often, but what about templates and generics? Both are used quite frequently.

I'm not convinced unqualified imports are necessary or helpful for generics. Zig has generics just fine without unqualified imports.

there's an article[0] addressing this topic though I'm not fully convinced

[0] https://narimiran.github.io/2019/07/01/nim-import.html

In short:

- You can do python style by doing `from x import nil`, then all imports are package qualified. (Con: style enformcenet)

- Ambiguous overrides are compiler errors in Nim (import has an 'except' keyword to omit some), so import overrides isn't as much of an issue. (Con: upstream package can break your compile by introducing the same function name)

- Dynamic dispatch works across modules so its nice to just do sqrt() instead of math.sqrt() and complex.sqrt() (Con: this feels meh to begin with).

Shrug.

The tl;dr is that Nim is statically typed so two functions that take different types are not ambiguous. If there is something that takes the same parameters, of the same type, in the same order it wont compile and you would use the module prefix to disambiguate.

In over 5 years of coding in Nim this came up once when two libraries defined a `Point` type. They were subsequently given distinct names so i didnt even need it for long.

Other options for ambiguous calls are renaming things during import with `import thing as othername` or using `type SdlPoint = sdl2.point` and so on, but i have never needed to do this.

The main point is that its a compile time error to try to call an ambiguous thing. Btw mouse over in VSCode shows you where a symbol is defined too.

Exactly, this is when this problem pops up.

Imo the best way to avoid this is to do: ``import sdl2 except Point`` and use the version of the procs that don't use Point, so you pass your own type as: ``render.drawPoints(p.x, p.y)``

I've been programming exclusively in Nim for 4 years, never had a problem with it. In fact it's a lot easier not to think were to get each symbol both in terms of refactoring and mental overhead. Good editor support means, with a mouse over any symbol, you can see were it was declared. But I don't use nimsuggest, or nimlsp, I simply never have to think about it, in my own codebase. I can understand it can be helpful when working in big projects like the nim compiler itself but I'm talking about goto definition not "qualified" imports, those are useless...

Nothing stopping you from using them in your own code. I prefer qualified too.

Note that this won't currently run on embedded devices. The ARC GC release with Nim 1.2.0 could be really useful on embedded processors the Async story hasn't been ported and contains cycles which leak memory. It works great ... until you OutOfMemory and it resets. The Nim core team is working on it. :-)

Still IMHO, Nim has a lot of potential on embedded devices. I've been using it to augment some C code I'm using on an ESP32 and it integrates pretty smoothly. It's enjoyable to write Nim code and I can do hacky C-like tricks with pointers if I need to which is great. Writing C isn't to bad until you want to just have some generics and containers like lists and hashes. C++ has them of course, but personally just to much headache for me.

Only obliquely related: I was hoping Nim would be the kind of language that preferred libraries over frameworks.

We have way too many frameworks and not enough libraries. A library comes with an API. A framework must arrive with its own architecture, a vocabulary to describe that architecture, tooling, a plugin ecosystem, configurations and a lot of best practices and recommendations. Are we paying too much for that initial, one-time effort of bootstrapping an app and stitching a bunch of libraries together into an app?

Just because too few people say this when the topic comes up: I prefer frameworks

At least for something like this. I want an opinionated framework that comes with a structure, an architecture and a philosophy. I want the right decisions to already have been made and I would like there to be one obvious place to put everything and one obvious way to do things.

I want to focus on my application and not be tempted to do lots of bike-shedding. I want to pick up someone else's project and instantly know where everything is likely to be.

Nim has a growing number of libraries that handle many of the features provided here. This is one of the first major attempts at a large, comprehensive web framework in Nim. It's not a signal of any sort of shift in the Nim ecosystem. Nor does the existence of one does not prevent or defer the existence of the other.

How many people are developing commercial software without a framework? Software needs architecture, tooling, best practices and the rest. Taking the library approach just results in ad hoc frameworks emerging over time.

I would think that the ability to assemble set of libraries into an application with a coherent architecture is a skill that is a part of a software engineer's basic training. It certainly was when I graduated. Has this changed significantly?

Virtually all C programmers, for one. Most Go programmers, too.

If I have a bunch of libraries that I always use (or at least 80% of them) to start a new project, isn't that a framework?

A framework is not a collection of libraries. You are still writing the top level code that glues all the libraries together.

A framework is that top level code, and it provides extension points or a pluggable system for the parts the framework author anticipated you needing to customize.

Works great within the boundaries, but gets very difficult once you want to step outside the framework author's preconceived notions about what one would want to do with their framework.

With the library approach, you have to invent some of your own conventions for routing and such at the top level, but you have a full understanding of it and can more easily extend it to handle new use cases.

Most projects outgrow a framework, at which point they tend to invent their own but it's a huge amount of technical debt to handle.

By definition, no.

In practice, maybe.

At work we make and use https://github.com/obsidiansystems/obelisk . While they're are certain framework-y things going on (we recommend our build system, our `ob` swift army knife command, etc.) nothing is obligatory and all the functionality is in separately libraries, and put effort to "bud off" ideas into separate repos etc.

I bring this is up as I think it's a good example of firstly absolutely not compromising on the end developer experience to compete with the monolithic framework, but secondly trying to beat them without stooping to their dirty tricks wherever possible.

Cute notion, terrible in practice. You end up in a MeteorJS situation where nobody knows what the hell to use, and new users? Good luck. You're toast.

This killed MeteorJS.

For Nim to succeed, it needs a killer framework like this. It needs to let developers build value for the companies they work for. I really hope Nim does grow tremendously, it's a great language.

Well, React was pretty successful... While being a library.

I don't know if this is a good example. While React tries to bill itself as a library, and there are arguments to be made for it being a library, there are equally compelling arguments as to why it is a framework.

If anything, this example does more to prove the distinction is semantic and trivial than anything.

This is correct; React is a framework.

Another cute notion but React is a framework in practice.

Please consider disagreeing with other people's "notions" in a direct, but civil manner. To call another person's idea "cute" is to demean his/her intelligence.

This looks nice, but I'd question what about this is "full stack"?

While full stack frameworks are unlikely to actually build at every layer of the stack, I'd expect them to be aware of and do something to address each layer. At a minimum I'd expect an ORM of some sort, something around validation of data from the client, bits for building APIs, potentially a structure for building frontend assets, tooling around deployments and maybe migrations.

This isn't what a lot of developers want, but it is full-stack. Maybe more things should be happy _not_ being full-stack and owning the idea that they are good at one part of the stack.

I’m surprised no one has mentioned Jester. It’s the web framework powering the official forum site for NIM.

https://github.com/dom96/jester

https://forum.nim-lang.org/

As a full time Python developer (though it's not my only tool in the toolshet) I've been looking at Nim for a while. I think it looks healthy currently. I'm wanting to give it a shot, and projects like Prologue are what I'm sort of waiting on before I dive in there.

There's a few things that concern me, like looking at the tutorial the echo statement is not a function, reminds me of Python before Python 3. Also the pascalCase function for readLine. I think I see why Python mainly uses single word functions for things, to avoid the debate about variable naming conventions as much as possible. Though now the community agrees on underscores mainly. Though this doesn't concern me as much so long as it remains consistent.

The `echo` statement is indeed, a function. The tutorial is taking advantage of Nim's optional parenthesis when calling the function.

  echo "Can you hear me?"

is the same as

  echo("Can you hear me?")

Nim also has somewhat unique rules about casing and underscores for variables and names. `readLine` is the same as `read_line` is the same as `rEadLInE`. However, the first letter must be the same (there are reasons for this), so `Readline` is not the same as all of those. The idea is that you can choose your preference and not have to think about what anyone else chooses. If a library uses pascalCase and your prefer snake_case then you can use snake_case.

This is a big point of contention for many people, but in practice, in my experience, it is never an issue.

echo IS a function.

What might be confusing to you is that in Nim all three following variations are the same

  echo("foo")
  echo "foo"
  "foo".echo

If you're curious about nim, I can highly recommend "Nim in Action".

looks good

Reply on news.ycombinator.com