Lwan, a high-performance web server with Lua support | Hacker News Reader

Lwan, a high-performance web server with Lua support

lwan.ws

99 pointsmattbostock55 comments

I'm certainly not in the market for a webserver with lua support, but if I were I would definitely be looking at nginx first. May I suggest that you make a subsection/table that delineates why someone would go for this rather than nginx? There is a section that says that if you think of apache/nginx first then that {nginx/apache} is probably best for you, but I think lumping in apache and nginx together may be a mistake, and even if I'm giving preferential treatment to nginx I may be swayed nonetheless.

I think it targets embedded machines which is actually quite nice in c + lua. openwrt uses this combo.

With a small code base there is also h2o and it already has http2 and a very good https performance (can serve around twice as much than nginx). After looking for most of available http servers h2o is my preferred one right now. Also I have a fork of h2o compiled as C++ (right now it cleans a bit redundancy/verbosity of C) and add SquiLu and Lua scripting (h2o comes with mruby only).

https://github.com/h2o/h2o

https://github.com/mingodad/h2o

https://github.com/mingodad/squilu

... however, my h2o servers get a lower score with Qualys SSL Labs in their default configuration than Nginx does. This is due to the fact that h2o seems to support ciphers per default that are considered less secure (according to Qualys SSL Labs while testing servers using h2o).

Interesting point, so it's using a dirty trick to show it as more performant ?

Nope. More insecure Ciphers are marginally slower than modern ones.

I've looked at h2o website and seems that someone asked this question and there is an example of a stronger configuration. Using it I did a test on a raspberry pi 3 and here https://gist.github.com/mingodad/e5a46c809ee030befc18 is the output.

What I noticed is that with nginx there is more connections even using keepalive (h2o 4 connections, nginx 102 connections).

h2o-default: 4123 rps

h2o-strong-256-first: 3849 rps

h2o-strong-384: 4233 rps

nginx-default: 2309 rps

I added also test with h2load, but with this tool h2o/ngix performs basically equal.

nginx-default: 1067 rps

h2o-strong-384: 1009 rps

Ha, I remember when we picked nginx because it was small, fast and simple years ago (compared to Apache). It is interesting how the cycle repeats. This is the new fast and simple, kind of what nginx started with.

Doesn't really work with nginx though, a webserver that it still extremely fast and simple.

Lwan, a backronym for "Lwan Will Annihilate Node.js". Cheeky.

Technically, I think that's a recursive acronym. Backcronym is when something has a name and you after the fact make it into an acronym.

According to the FAQ it was suggested a posteriori, so technically it's a recursive backronym!

"Does it support HTTP2? -> Not yet"

Ok, back to Nginx/H2O/Caddy.

Algernon supports both Lua and HTTP/2. Written in Go.

http://algernon.roboticoverlords.org/

Linux only - bummer. Not even FreeBSD.

The turbolua.org project is somewhat similar (i.e. a webserver with Lua support).

Replacing the epoll() with kevent() looks pretty straightforward if you want to try it out, but FreeBSD should be able to do other tricks as well (e.g. http filters)

It builds on FreeBSD (soon OS X), but it's not working yet.

Which is why most "competitors" go for libevent (or libuv), as that abstracts over all the various select(3) improvements.

Turbo does not just support Lua, it's actually written with it.

It seems like a cool project with a nice clean coding style, but the lack of API documentation makes it less inviting. I'm sitting here grepping through the repo without much of an idea of what to try next besides reading the whole code base.

Is Lwan a good project for a C beginner to learn more about C and contribute?

It has been made with great deal of attention to details and code quality, so yes, it is a good one.

Finally a nice, open-source answer to Barracuda:

http://barracudaserver.com/

Or closer to one, anyway. Support for RTOS's like QNX and INTEGRITY puts it ahead in a big way. Someone might port this to GenodeOS, MINIX 3, or something similar. Nonetheless, it's a start that might save someone some money. If it's well-done internally, it might also be easy to port to other languages or secure CPU's in development. That rids us of need for garbage collection or JIT.

Interestingly, Lwan is the fastest server for json serialization : https://www.techempower.com/benchmarks/#section=data-r12&hw=...

This could be a great choice as a json server, if they showed more examples working with data stores (postgres or cloud data stores) and also testing

Is the name a take on that phony gwan server? (It seemed fast, but it absolutely did not respect unique sessions when I bench'd it.)

Kind of. I've tried making the page just as phony, but that's more difficult than it sounds.

I started out using Apache and now I am on Nginx. I like the simplicity of the configuration of Nginx along with the great documentation and modules.

It would be a hard sell for me to switch to Lwan. My sites are not getting 300k connections all at once.

Interesting. Someone used lwan as a replacement for the Node.js HTTP server https://github.com/raadad/node-lwan

The C API can be used from Lua with LuaJIT, right? It seems like this is the way to go to create the bindings.

It's possible with FFI, yes, but it's not the way it's currently done in Lwan.

The C interface looks really well made. I might try and play around with this to make a few microservices.

> Hand-written HTTP request parser

Aren't they all hand-written?

Yea, I'm not sure why that's a benefit, either. It's like they're trying to sell stuffed animals on Etsy, "hand sewed."

That's not the only joke in the feature list (or the whole page, to be honest.)

A few years ago Ragel was all the rage: http://zedshaw.com/archive/ragel-state-charts/

More or less. There was a minor parser combinator arms race in the Rust community a few months ago using http parsing as the benchmark and my takeaway (only half paid attention, I haven't needed to write a parser) was that they got to roughly as fast as handwritten C parsers. Example:

https://github.com/Geal/nom_benchmarks/tree/master/http

This hasn't translated into actual use in the Rust HTTP libs with hyper and rotor-http using their own handwritten parsers but I still think it might.

I think the easy c interop is kind of neat.

Is Lua the new in vogue programming language?

Lua has been the standard extension language in gamedev and adtech for years and years.

It was in the 90s, especially after the Dr Dobbs article about it, and tons of late 90s and early 2000s games, like Grim Fandango, Baldurs Gate and Escape from Monkey Island used Lua due to its hotness.

Actually, Grim Fandango was the very first known game to use Lua and wasn't due to any hotness. Bret Mogilefsky, who against skepticism, dared to try to create a new scripting engine to replace the aging SCUMM engine. He was given the Dr. Dobbs article by a colleague and things came together. After the technical success of Grim Fandango, it was applied to Escape from Monkey Island.

A few years(?) later at GDC, there was a 2 hour session to discuss scripting in games. Near the end, after everybody was depressed with no viable solutions, Bret's colleague exclaimed, "Or you can do what Bret did" and he was dragged up on stage. Bret gave an impromptu spiel about what Lua was and how they used it at LucasArts. Everybody else frantically scribbled down notes and that was how the video game industry adopted Lua as the dominant game scripting language.

*Source: Bret Mogilefsky's talk at the very first Lua Workshop in 2005

Lua is a simple and only slightly quirky (1-based arrays) dynamic programming language. It has exceptionally small and high quality implementations in Lua and LuaJIT. It has very good documentation.

Its main downside is that the language changes in backward-compatible ways between releases. And worse, LuaJIT has in some sense forked the language. Lua 5.3 introduced integer support, but there are no plans to add this to LuaJIT AFAIK.

The 1-based indexing isn't particularly bothersome. It's the method vs. function determination at call site combined with default nil for unspecified arguments that really gets you.

I've never had any problem with the two things you mentioned.

> default nil for unspecified arguments

Isn't this what most scripting languages do? I know it's the case with Python and PHP.

And I disagree about the 1-based array indexing. With algorithms that require a lot of 1-off values, it can be a huge pain in the ass, particularly if you're trying to integrate it with C (and its 0-based array indexing). Still not sure why the creators chose it, given that Lua was designed for easy and close C interop.

Why? From one of Roberto Ierusalimschy's talks (starts at slide 6):

http://www.inf.puc-rio.br/%7Eroberto/talks/ws2011.pdf

Excerpt:

- Not all languages are 0-based: Icon, Fortran, AWK1, and Smalltalk are 1-based; Snobol, Pascal, Modula, Modula-32, and Ada have configurable bases.

- Currently, many languages are 0-based due to influence from C.

- Ironically, none of them share the reason that made C 0-based (where a[e] means *(a+e)).

1-based arrays:

- Much more intuitive: first is 1st (not 0th). ISO-C: “E1[E2] designates the E2-th element of E1 (counting from zero).”

- Much easier for non programmers. Easy for (good) programmers :)

- Historical reason: Fortran used 1-based arrays, and most first users of Lua had a Fortran background.

The presentation asserts that 0-based arrays have "More interesting mathematical properties". Dijkstra's classic paper on this is a must-read: http://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF

> Isn't this what most scripting languages do? I know it's the case with Python and PHP.

This is false for Python:

>> print bla

NameError: name 'bla' is not defined

In Lua:

> = bla

nil

That's yet another source of nil, but it can be fixed with modules like strict.lua, or simply

    setmetatable(_G or _ENV, {__index = error})
    -- `error` for the example, you can write a function that prints a better error message.

Python errors out if you don't specify all arguments.

    >>> def f(a):
    ...   pass
    ... 
    >>> f()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: f() takes exactly 1 argument (0 given)
    >>>

I agree 1-based indexing isn't ideal, but in the grand scheme of things, the default nils and the method/function syntax are a constant source of mistakes and frustration for me. In Lua you have to call `obj:method()` which to pass `obj` as the first argument. If you accidentally do `obj.method()` - there is no error, but the first argument becomes `nil`.

In Python, there the call syntax is consistent and `obj` is automatically passed as the first argument for methods.

> but the first argument becomes `nil`

s/first/last/, the arguments will be shifted to the right. But yes, it can be annoying.

Thanks for the corrections. I've been switching a lot between Lua and Python for the past year and have clearly gotten some things mixed up.

Speaking to its small and high quality implementations, Verisign gave a talk on how they used it for their high availability/performance servers. Rather than the usual throw more hardware redundancy at the problem, they went the other extreme and audited and understood every line of code in Lua so they could create a reliable system by eliminating bugs.

And Wikipedia (Wikimedia) uses Lua for their multimedia scripting engine. Besides good performance, the implementation was good and small enough that it could be audited for security.

I'm interested in it because the machine learning library Torch is using it (and by extension Facebook and other big tech companies). It's also one of the fastest dynamic languages for web applications

Reply on news.ycombinator.com