A fast and static web server for serving web apps | Hacker News Reader

A fast and static web server for serving web apps

shortfin.io

77 pointstimothy89130 comments

  Just run the following command as root to install the server.

  # wget http://shortfin.io/install.sh && sh install.sh

What? Are they serious? Is this becoming a thing? Please don't tell me that this is a thing.

This has been a thing since the mid-90s when you could execute 'telnet sci.dixie.edu 1 | sh' to install an IRC client. It preys on the clueless and one of these days, some project that uses this install method is going to get owned and a bunch of machines will get malware.

Then again, most of us blindly run what we assume to be GNU autoconf scripts and never read Makefiles before we 'make install' as root. Nor do most of us read or audit source code, even for security-sensitive applications.

a lot of times it's curl instead of wget.

I don't see any rational basis for this. If you weren't vetting the code you downloaded before, you don't lose anything by blindly running the commands on one line instead of three. And if you do want to vet the code, you know what to do. Practically no installation instructions include a "vet the code" step, so you aren't losing anything here.

The lack of TLS is worrisome, but that's not so much "becoming a thing" as it is "a bad thing we're just finally starting to get rid of."

Although this isn't new, it should make you throw up in your mouth a little bit.

Malicious compromise is not the only way that an install script can become broken, or corrupted, on a remote site. The ability to checksum what you're running is a very useful sanity check and you should be doing it with all downloaded scripts/packages.

I suppose this is tilting at windmills, though.

> Malicious compromise is not the only way that an install script can become broken, or corrupted, on a remote site. The ability to checksum what you're running is a very useful sanity check and you should be doing it with all downloaded scripts/packages.

Yes, and you still can do that. But the standard installation instructions almost never include it. For example, nginx's build instructions boil down to "run configure, possibly with some flags, and then run make and/or make install". That includes the exact same vulnerability you complain of here — it never tells you to checksum anything. This is true for essentially all the software I've ever downloaded. So again, still not "becoming a thing."

The assumption is that if you're the sort of person who customarily checks code before they install it, you know how to inject that step.

>"If you weren't vetting the code you downloaded before, you don't lose anything by blindly running the commands on one line instead of three."

Nginx is included in the OpenBSD base system (as of 5.2) and it does get vetted.

I am not seeing the relevance.

Your assertion is that if we want to vet the code, there are steps to do that and no conventional installation includes these steps, correct?

Well, getting the "trust us, this is easy" type installation instructions here isn't terribly comforting from a brand new project. That's the sort of thing I'd expect from a source I know. Not from this project that I didn't even know existed before this post.

It's unrealistic to expect those who take security seriously to do so in the first place. Regardless of how often it's done elsewhere.

OTOH, there's a well known and trusted software distributor that does the vetting for you, which this project doesn't have, and I'd much rather do the "vetting-skipped install" if you will for something they distribute instead.

That's all true, but I don't see what it adds to the conversation. It's just a plug for nginx. "Conservative, security-conscious people like older projects with large, trusted organizations backing them" is something of a tautology.

This conversation started when you said you don't see any rational basis for the criticism of running wget/install as root when no vetting takes place on conventional installations. My post wasn't a plug for Nginx, but an imploring to know what you install.

Since I can't know everything that's running on my system, I'd trust that from some place that actually does vetting. It doesn't need to be OpenBSD, but some competent source with an established record for security.

That's not tautology. It's sane practice that I wish more people followed.

The wget/shell install as root is presumptuous in the extreme for a young project with no history. In fact, many young projects, whose security we know nothing about, have similar installations and that's a bad precedent to set and continue.

Yes, but at this point, of course you won't want to use the OP's software, or any other newish project linked on Hacker News. The method of installation (which is what we were talking about here) isn't the problem, because you wouldn't use anything from him because he isn't OpenBSD. And that's OK — you can be as conservative as you like. It just had me confused as to why you were bringing it up.

  because you wouldn't use anything from him because he isn't OpenBSD

My sarcasm detector must be malfunctioning. Who said I don't want to use his software or install anything linked on HN because they're new? I've checked out and installed plenty of things linked on HN, not because they're new, but because they're interesting. But lax procedures like the root shell install aren't good indicators.

In fact, I was just going over his install script to see whether it's worth installing and test, but seeing things like this :

  # The umask is ridiculous, or mkdir does not conform to POSIX,
  # or it failed possibly due to a race condition.  Create the
  # directory the slow way, step by step, checking for races as we go.
  
  
  case $dstdir in
     /*) prefix='/';;
     -*) prefix='./';;
     *)  prefix='';;
  esac

Is what makes me nervous.

> My sarcasm detector must be malfunctioning. Who said I don't want to use his software or install anything linked on HN because they're new? I've checked out and installed plenty of things linked on HN, not because they're new, but because they're interesting. But lax procedures like the root shell install aren't good indicators.

Oh, my apologies. I think the context of your other comments caused me to misunderstand that one. Your initial comment was simply "Nginx is part of the base OpenBSD install," and I read your later comment as an explanation of that, which made it sound to me like "I'm OK with nginx because it comes with OpenBSD and not OK with the OP's server because he's not a trusted organization." I understand better now.

Accepted :)

And I apologize as well. My posts were venturing over to the pushy/rude side and I don't want that to be the case.

Truth is that I do enjoy seeing new projects like this because it forces me to look at things with a fresh perspective. Yes, the install method did give me some pause, but maybe I've been spoiled by things that I'm comfortable with so I welcome the chance to pour over the code for something totally different.

Yes, it's a thing, and aside from the lack of HTTPS it's not much worse than your standard "./configure", "make", "make install" install flow of the last 20-whatever years of open source. If you don't trust the developer to serve you a safe install.sh script you shouldn't trust the rest of their code either. If you don't trust them you should audit the code no matter what the install process looks like.

That said, if you're going to do this, you should do the following:

1. HTTPS only. No excuses.

2. No URL shorteners, which make or may not use HTTPS, and introduce another point of failure.

3. Ideally serve directly from Github repo. Eliminate another point of failure. The user can be reasonably confident that the install script is the same one that's in the Github repo.

It's a lot worse than the standard configure, make and make install.

For one, it's only requiring one simple script to get compromised and replaced to compromise your entire machine. At least when I download the tarball, unpack it, etc. I've got several steps before I even get to the make install that I would run as root. And I might not even do that depending on my install target. If I do, I have ample opportunity to verify the package in question and examine the script I'll be running via sudo before I do so.

With the instructions provided by this site, not only is there no opportunity to verify the script I've received is the one I wanted to (HTTPS alone is not enough for this, the site or domain may have been compromised) but they're instructing me to just run it immediately as root without any examination.

This is a terrible practice, and while yes it has been around for a long time in the past it was something only done and encouraged by idiots, it seems to be gaining popularity.

This is a bad idea, stop doing this people.

Provide an install tarball and provide a checksum of it that we can verify.

"Provide an install tarball and provide a checksum of it that we can verify."

https://github.com/timothyej/Shortfin

My point was the people who are going to blindly run curl/sh commands are also going to blindly download the source tarball and run configure/make/make install, and the people who want to do more verification are certainly free to do so.

I wonder how much work it would be to modify that one-liner to include an MD5 or SHA1 check? Of course it might then be shell and OS-specific...

Unless curl or wget can confirm the sha1? Oh... found it! http://superuser.com/questions/453823/is-there-a-tool-that-a...

Without HTTPS in the main site that is mostly useless

True. Though for a bit of benefit, if the download was hosted separately from the site, that'd be two things to hack. The downside, of course, is that script updates would require new SHA1s. It would be best, then, to insert version numbers into the installer scripts, and a version check. That way if a new version comes out, the installer could warn about it (y/n), while still verifying the SHA1.

Yes people are going to dumb things. But that doesn't mean you need to encourage and facilitate it. That's my point. If you know better you should be facilitating the proper way to do it, not the worst way.

Teaching people good security practices is a lot more involved than telling them to run 3 commands instead of 1.

I'm a Windows user. Virtually every piece of software I install comes in the form of an executable, that I download-and-run directly from the browser. I often give this executable administrator priviledges. The executable can then do whatever it wants on my computer. Usually it will install said piece of software (except when it was made by Oracle, it'll install said software and the Ask.com toolbar)

This has been standard practice for decades. It's very comprable to that wget oneliner. Why is it bad? Or, why is the wget oneliner bad but not Windows installers? Do Windows sysadmins compile tarballs and verify checksums? Or are Windows sysadmins simply dumber/less paranoid than Linux sysadmins?

You're giving those executables full control over your computer. Yeah, you won't get burned very often (depending on where you're getting these executables.) But when you do get burned it's going to be catastrophic. That's the problem.

It's a bad practice.

Hm, I see your point.

How would you then recommend installing non open source software on a production system? E.g. a freeware (but proprietary) FTP server or a database engine.

If you are taking recommendations, I wouldn't recommend installing them at all. You could ask the authors to support a platform which has a better security model.

(Given that there are open-source and robuster-than-Windows platforms freely available.).

Cool, I'll just email MS and ask them to port Sharepoint to BSD.

Good job making an honest and good recommendation sound naive.

How is 'just ditch Windows' an honest and good recommendation?

That was a terrible recommendation. Proposing a solution when you know nothing about the situation nearly always results in a pointless solution.

Suggesting that the best way to securely install Windows packages is to use a different operating system smacks of immaturity, and is obviously not a practical solution in Windows environments.

A better suggestion might be: If you do not trust it, run it in a VM/sandbox.

That might be a good idea, assuming they do listen to user feedback.

Chrome for Windows installs to a user's own app directory. It doesn't require admin privileges.

1. Don't install software from untrusted sources. 2. The trusted sources should provide checksums so you can verify the package before installing it.

This protects you from a compromised package so long as the page you accessed the package from has not been compromised (if that page is compromised, the checksum could be changed to the new one).

I have no idea how you would verify closed source software if the checksum and package are both corrupted...

"2. The trusted sources should provide checksums so you can verify the package before installing it."

If the checksum is published to another trusted location, in addition to where you download the files, that would help in the case that an attacker compromises both the checksum and the package on the download site.

The developer could post the checksum to Twitter at the same time as making the package available for download. An attacker would need to compromise both Twitter and the download site.

Blindly installing windows executables and giving them admin privileges is at least as stupid. This being common practice is probably the primary reason botnets are easily able to exist. This being common practice is why windows sysadmins disable users from having admin privileges. Because apparently you're too dumb to be trusted with them.

>>>>Provide an install tarball and provide a checksum of it that we can verify.

This 100%.

When I first started experimenting with Linux after being a lifelong Windows user, this was the same advice I got from a seasoned Linux developer: ALWAYS COMPILE FROM SOURCE - NO EXCEPTIONS.

He just told me even though Linux is secure and not as big a target as MS, he said you want to establish good habits from day one and not get lulled into a false sense of security.

Why? If you don't trust your Linux distributor then you need to find a different one, but I don't think the answer is compiling everything from source; specially if the upstream project doesn't support 100% your target distribution.

See: http://www.debian.org/security/

What? You don't need a tarball to be able to verify the package via a checksum. In fact, a good packaging system will do checksums automatically, plus offer public key signature verification. I know deb and rpm files do that, and I believe many Windows packaging systems also provide checksums and public key signing. So do apt, aptitude, yum etc. On most of these systems, you even have to manually disable this option if you don't have the repo's public key.

In fact, of all the install methods I use regularly as a Linux user, the only one that does not offer automatic checksums and public key signing is make. You can certainly verify checksums and public key signatures, but you have to do it manually.

Do you look through all the code you build from source? Highly unlikely. If not, what's the point?

Security is a lot more social screening than many people in the tech world would like to think. Know and trust the people you are running code from. If you don't know and trust them and still need their code, that's when you want to spend a couple of hours reviewing the source.

ALWAYS COMPILE FROM SOURCE - NO EXCEPTIONS. And still you can have problems if the source package has been compromised: https://news.ycombinator.com/item?id=6167000 [ok it's javascript] http://www.exploit-db.com/exploits/15662/ [source code -ProFTPD] http://www.cert.org/advisories/CA-2002-28.html [source code - Sendmail]

etc..

or do you want to tell us that you inspect every line of the source code before compiling something?

I think he told me this so I would get into the habit of doing it. I was so used to just clicking on an executable file and just installing the program without ever thinking it may contain malicious code.

I can tell you I don't do this every time. Now if I'm installing a 3rd party extension or plugin, I usually do it in a sandboxed environment, run some pen tests on it to check how secure it is and see if there's any malicious code before I install it on my machine or server.

And yes, even then, it is still possible to install malware on your machine.

No, they're not depriving you of any opportunities to verify. They're offering you a shortcut if you decide that you trust them enough to forgo examining thousands of lines of code. If you decide you want to review the code, just wget it down and then install it after you review it. And if someone doesn't know enough command line to do that, are they really going to be in a position to audit the code?

If the site or domain is compromised, wouldn't a mildly competent attacker change the hashes also?

It's not "curl | install.sh" vs "configure/make", you know? What about using any package management system?

It's not the same as configure,make,make install as you can always: configure --prefix=$HOME/whatever

and later run it on some unprivileged TCP port over 1024.

The install.sh goes off to github and pulls the latest - it's not a method for keeping the same version across machines. It may be fine for your needs, but it'd still be nice to have 'stable' or suggested versions to use.

This has indeed become so established that chocolatey.org can offer the following command with a straight face and call it "Easy Install!":

  @powershell -NoProfile -ExecutionPolicy unrestricted -Command "iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))" && SET PATH=%PATH%;%systemdrive%\chocolatey\bin

The request parsing code seems to be of dubious quality https://github.com/timothyej/Shortfin/blob/master/src/reques...

From a cursory glance:

1. https://github.com/timothyej/Shortfin/blob/master/src/reques...

should be (data_len - i >= 4) since it accesses data[i+3]

2. https://github.com/timothyej/Shortfin/blob/master/src/reques...

shouldn't headers[header_count]->key also be null terminated?

3. https://github.com/timothyej/Shortfin/blob/master/src/reques...

header_count can become greater than 49 which causes a heap overflow at https://github.com/timothyej/Shortfin/blob/master/src/reques... and https://github.com/timothyej/Shortfin/blob/master/src/reques...

I'll look them up, thanks for noticing!

the code in general is not that great, imho.

as well as the header count (which i came here to post), there's another suspicious hard-coded size limit in the number of servers (1000). although you could only crash the system in that case by configuring too many.

there's very little error handling. good c code returns error codes all over the damn place. this hardly has any.

i wouldn't use this.

What's a "static web server"? I haven't seen this term.

A webserver for only static pages, i.e. it won't talk to a runtime? But it's "for serving web apps", this means all the dynamic content is pulled from client side JS? Then you would still need programs on the server to answer AJAX requests.

I did not find an explanation on the page or by Googling.

A server that only serves static files. It doesn't enable the use of dynamic content like CGI[1]. Compare this to something like Apache or nginx. Both can serve static files but also support a number of ways of either directly running scripts (ex: mod_perl or mod_php in Apache) or proxying to other servers that handle dynamic content.

[1]: https://en.wikipedia.org/wiki/Common_Gateway_Interface

OK, then, sorry if I'm being dense, but how can it be "for serving web apps"? App seems an odd term for a site consisting of only static files.

Perhaps you've heard of this new language, JavaScript?

Still not much of an app unless the JS does AJAX, but then it's talking to a server that's running some dynamic process, maybe with DB access etc. - right?

Not necessarily. You can make an ajax request to a static resource. The server doesn't have to be a dynamic process. Static resources are just files.

A client-side JavaScript Twitter could be done this way. Every tweet, or timeline can be pre-baked into a static json file, and ajaxed in.

Twitter can then be simplified to a backend application that updates these files.

Granted, tweeting is a separate matter; that requires an endpoint that processes data.

Cross-domain REST is the big win. MongoLab, APIs, Widgets, etc.

thttpd is probably the most well known "simple" webserver, and is very, very fast[1].

In fact, I think the op needs to explicitly mention thttpd on the page for shortfin and provide a performance comparison, since that's all anyone really cares about ... "is it faster than thttpd ?"

[1] http://acme.com/software/thttpd/benchmarks.html

The citation should include a caveat that the benchmarks are from 1998 and that they were ran on a Pentium 100Mhz.

The notes very clearly state, "The benchmark test system is a 297MHz Ultra Sparc with 256MB RAM / 512MB swap running Solaris 2.6".

What you're referring to is merely an observation about the actual capabilities of web servers in general at the time: "Most of these servers have enough oomph to keep a T1 saturated, running on a lowly 100MHz Pentium or maybe even a 486."

One big thing with this is to develop a web app consisting solely of static files.

The trick is that most of the dynamic stuff you get from REST services on OTHER servers.

For example, people maintain static blogs with Jekyll and can enable commenting with an HTML widget.

Or your app can use MongoLab for it's back end data which again is a separate server.

It turns out you can do almost anything just developing with your own static files.

This is one of the reasons hosting prototype web apps on DropBox is popular.

Close.

Can add this to the list like thttpd.

Something I keep thinking I'll build is a fast, high connection count, limited HTTP server, something that is essentially a wrapper around a program that works like 'regular' and emits HTML. It is a corner case in a custom corner, but the target it something which is essentially a 'transponder.'

In the 'Internet of Things' I want to build a wrapper/environment such that my program can be

  main(int argc, char *argv[]) {
    uint16_t sense;
    uint16_t chan;
   
    sense = adc_read(atoi(argv[1]));
    printf("Sensor %d reads : %d\n", chan, sense);
  }

And then link my 'wrap around web server on it' and then it can be accessed with wget 'http://ip:port/?1' have it do the right thing.

The keys are low memory footprint, lots of connections, easily wrapped around a 'regular' program.

"easily wrapped around a 'regular' program"

Sounds like you want CGI :)

Yes, "static CGI" if such a thing was a thing :-). I've got a thttpd tree that I do this sort of hack in (these are great for monitoring large numbers of servers for example).

I don't understand (and am interested). What's "static" about it? With Apache and mod_rewrite and CGI, I could make the functionality you want (which is, of course, completely not low footprint, but I don't understand how what you want is different from CGI).

Static in the sense that it always executes the one function (doesn't load it from disk, its part of the executable) and so there is never any risk that some other path might get you 'out' of the docs directory and into the cgi_bin directory. So in this case static is code for 'compiled in' versus 'dynamically loaded.'

Like a statically linked HTML document? I know you're talking C code, but I could imagine some kind of HTML to C "compiler" that could you then statically link into an executable. Then you could have a full website that ran as a single static executable, including CGI-esque calls.

It's actually rather easy: http://www.linuxjournal.com/content/embedding-file-executabl...

That's cool! I wonder if this would be (much) faster than just letting nginx or this embedded httpd read the files directly and serve them. I would hope that the OS or server would cache things enough to make it unnecessary, but it would be interesting to find out.

HTML isn't that big, so if you used linked resources, it would be pretty simple to make a C program and compile it to a binary for any purely "static" site.

That's how the Kore web server works ( https://kore.io/ ); it makes you generate a static.h file at compile time that contains all of you your static content.

But like you say, I have to wonder if it actually makes a difference performance-wise. The kernel will cache the static file data anyways. The only improvement I can see is maybe to avoid having to look up file metadata (length and mtime)? Or to avoid a syscall?

Openresty is a good option for this - Lua in nginx.

The apache module to do this is like a screenful of code. Really, mod_perl or mod_python or mod_wsgi can do this with dynamic languages, the code is only loaded once (per child in prefork apache). Your apache config can avoid reading the filesystem all together.

As long as you have a web server that can load modules and be invoked on specific URL requests, you can embed this.

I'm more familiar with Windows, but Linux should be similar. If your application is executed often, then it's pages should be in memory already, and not necessarily need to be read from disk. In terms of execution, if it's a compiled binary, it should be fast enough, if you need more, what you want is a lightweight application server anyway.

The problem with regular old CGI is that for every request the web server gets it has to run some binary file and setup a new thread for that every single time. Also, that's ignoring the filesystem overhead which isn't negligible even if it's already cached. You're basically turning your web server into a glorified shell script and even though all of your CGI scripts are compiled it's still horribly inefficient. FastCGI on the other hand is basically like giving the web server a socket to a persistent process that runs the associated code in an existing thread instead of having to spawn a new one on every request.

Basically FastCGI is like having another server just for your dynamic content while the web server sits in front of it and takes care of hosting the static content and deferring to the FastCGI server when appropriate.

I'm specifically saying that the File System overhead is negligible for files/executables that are frequently accessed. Windows uses RAM to cache these files... Setting up a Process to execute said file vs a service that stays running is a different story.

Sounds like you want AOLServer.

http://www.aolserver.com/docs/devel/c/c-overview.html

I had an idea like this when I thought about creating a small server on an Arduino that I can use to control/get status updates for a pellet stove for my cabin (TBA).

Luckily, there's already code for that, in case you're interested : http://arduino.cc/en/Tutorial/WebServer

I've used Snorkel[1] for a few C/C++ projects in the past, you may want to check it out; it sounds like it could be what you're looking for (or quite close). Link it with your code, pass it a few config parameters and let it serve static files -- or you can overload GET/POST functions to provide dynamic functionality.

1: http://snorkelembedded.webs.com/

That looks interesting, if somewhat abandoned.

I don't get it. You create a server to solve C10K problem, and then host its own website with nginx? Why?

Having HN crowd coming and testing your server would be a great test, no?

The actual site is down for me too, http://shortfin.io:88/

The link works fine for me.

Work firewall, duh, sorry.

I think in this case nginx is being used as a reverse proxy.

From the footer:

> It was created in 2011 by Timothy E. Johansson and is the base for the send-API and the tracking server for AlphaMail.

> It also powers this website (though behind nginx). You can reach it directly on port 88: shortfin.io:88

I read that. But why?

What's the point in using this if you're going to have nginx in front?

I suppose he is hosting other stuff too, and nginx proxies.

I host more websites on the same server and as kajecounterhack said I'm using nginx as a reverse proxy. The server is fast and have no problems at all with the HM crowd ;)

What are advantages over nginx? Memory footprint, higher through-output?

I mean, what's the main "selling point". Why should one use this instead of battle tested nginx?

Don't get me wrong, I'm not criticizing, I'm just curious.

On my tests it's a few 100 requests faster per second than nginx but the average should be around the same. The kernel is the bottleneck in this case I believe.

There isn't any "selling point" ;) Anyone could use it as they like. The main usage for me has been in other projects where I needed a fast and lightweight http server. Such as tracking servers etc. It's pretty "experimental" but I think I will use the "original" server to serve a web app that I will be launching soon. The problem for me has been the lack of a reverse proxy.

But why do you need reverse proxy at all if your server is as good as Nginx? And if the speed is comparable and Nginx has more features why did you write your server then?

I've never stated that it was as good as nginx?

The answer to "why" is answered in these comments:

https://news.ycombinator.com/item?id=6168052 https://news.ycombinator.com/item?id=6168541

If I understand, it's "lighter" than Nginx. Why not writing how much lighter and in which aspects? Of course I can compare the code bases, but you know exactly as you wrote yours, so it would be really nice if you'd write what you know. Thanks.

I was going to say that there are *~ files in the repository, they should tidy a little bit because things like that don't look good.

Then I thought may be I was being a little bit to picky and unfair, so I checked the code of some random files and... well, it looks like it was part of a learning experience but it definitely needs work before being considered a viable option.

>wget http://shortfin.io/install.sh && sh install.sh

Isn't it recommended that any project doing one-line installs use HTTPS?

In the same way that it's recommended for any other program, yes.

Side-note for the author: the download can be made more elegant that way

    $ curl http://shortfin.io/install.sh | sh

I definitely agree with the HTTPS. For example this is what meteor proposes:

    $ curl https://install.meteor.com | sh

asdfsdfgkjlsdfg!

Streaming your install script directly into computer execution without even giving it a once-over, md5 compare, etc is atrociously insecure.

This seems to be a common complaint but I can't see any reason you can't give it a once over when it's hosted on the web…

That's completely orthogonal to what we're talking about here. If you're already not doing it, you're not doing it any less by being more efficient. And it doesn't really matter if you're running it in a throwaway VM like I do.

    curl https://install.meteor.com  | less

... and especially not if you're suggesting run it as root. You should never run anything as root unless you know what it is. Having it over HTTP vs HTTPS makes things really bad though.

I just ran two speed tests since there aren't any benchmarks provided

1. First Test is using http://www.webpagetest.org/

Results shortfin.io:88 - http://www.webpagetest.org/result/130806_71_13HK/

  First View: 1.945s
  Repeat View: 1.632s

Results shortfin.io - http://www.webpagetest.org/result/130806_QV_13F7/

  First View	2.099s	
  Repeat View	0.084s

2. Second Test is using Apache Bench

Results (Best results of 3 runs): ab -n 100 -c 100 http://shortfin.io:88/

  Time taken for tests:   12.125 seconds
  Requests per second:    8.25 [#/sec] (mean)
  Time per request:       12124.501 [ms] (mean)
  Time per request:       121.245 [ms] (mean, across all concurrent requests)
  Transfer rate:          38.46 [Kbytes/sec] received

  Connection Times (ms)
              min  mean[+/-sd] median   max
  Connect:      210 1114 1517.2    360    4041
  Processing:  5017 8407 2034.0   8715   11762
  Waiting:      169 1084 3054.8    188   11408
  Total:       9057 9521 1068.7   9058   12123

Results (Best results of 3 runs): ab -n 100 -c 100 http://shortfin.io/

  Time taken for tests:   5.790 seconds
  Requests per second:    17.27 [#/sec] (mean)
  Time per request:       5789.949 [ms] (mean)
  Time per request:       57.899 [ms] (mean, across all concurrent requests)
  Transfer rate:          82.97 [Kbytes/sec] received

  Connection Times (ms)
                min  mean[+/-sd] median   max
  Connect:      171  249  35.7    263     290
  Processing:   172 1798 2345.5    212    5524
  Waiting:      170 1055 1825.1    212    4949
  Total:        344 2047 2368.2    469    5789

You need some better hardware

$ ab -n 100 -c 100 http://shortfin.io/

Requests per second: 1363.85 [#/sec] (mean)

Benchmarking a fast remote static server is pointless.

You're basically benchmarking your internet connection.

I think the landing page would greatly benefit from two short paragraphs:

Why Shortfin when we have Nginx/Lighttpd?

Why Shortfin when we have thttpd?

1. Shortfin is not a replacement for nginx. I use it myself to create other projects, e.g. fast API servers with one purpose only and a REST server that uses redis as storage.

2. See #1.

3. I don't try to sell anything, it's just an open-source project that I think maybe someone could benefit from. I've learned a lot while coding it.

OK, Shortfin is just trying to be an efficient HTTP server, without a radical differentiating feature or implementation approach.

Well, let a hundred flowers blossom.

While I appreciate the time and effort that went into this, I'm more concerned with security than just speed alone. It's still relatively easy to throw hardware at the load problem (up to a point, naturally), but safety shouldn't take a back seat to performance.

The wget shell script installation does make me nervous and I'm glad the source is available separately. Blind installation of scripts was never "a thing" with me.

Of course, I'm still glad people are writing proper web servers (as opposed to simple 2-10 line ones). That creates an opportunity to explore the field with fresh ideas.

I find it very hard to trust any C code without tests. Is this a thing for C code? Why most of the new projects in C lack tests?

My favorite part is this, under the config section: # vim /etc/shortfin/shortfin.conf # /etc/init.d/shortfind restart

No explanation of what the configs are, just that "its important to configure it right."

Why not put this information on the site?

I can't find anything related to the response headers the server can send to the client, and the OP site just serves responses with content-length, content-type and server. Anyone?

This code is far off from being production ready. Skimming through the code quickly, there's barely any error handling. HTTP parsing is not compliant either.

https://github.com/timothyej/Shortfin/blob/master/src/respon...

I am not sure what's the point of using it instead of nginx.

There's a sample configuration at https://raw.github.com/timothyej/Shortfin/master/config/shor...

You're from Lund! I've been studying in Lund!

By the way, can you show some statistics and performance metrics with respect to nginx, Apache and such?

To put it another way: why did you choose to write your own webserver instead of using, say, nginx?

I am curious too, on the performance metrics. He is still using Nginx though.

Nginx is just acting as a reverse proxy because I host other sites on the same server. Benchmarks will come!

Lund is awesome! ;)

I will perform some benchmarks and compare it to nginx (and maybe apache) in a while.

By the time (2010, 2011) I was working at a company that needed a tracking server for tracking clicks and views in email. So I started to mod lighttpd and thought that I could make it faster. So in true "challenge accepted" spirit I started to write my own HTTP server. And since then I've made it better and better. It's not meant to be an nginx killer. It will always be very basic, fast and lightweight. It's great for projects where you need a web server included.

As I wrote in another comment I have used the code base myself in various projects such as:

* Tracking clicks, view, etc. for AlphaMail * The send-API server for AlphaMail * A DMARC report server (https://github.com/amail/comfirm-dmarc-report-server) * A REST server with redis as storage

I've never used it as a static web server in production yet. Mostly because of the lack of a reverse proxy and other great features that nginx has and shortfin not.

okay ... why? what's special about this? features? benchmarks? anything?

Why not?

The author claims it's 'high-performance and open-source' - sounds good to me. It's not as if we have a hundred web servers all competing as we do static blogging systems, the more web servers the better in my opinion, just as with web browsers.

The zip file is 312KB as linked from here, so it's pretty compact, possibly suitable for embedding in apps, etc:

https://github.com/timothyej/Shortfin

A lot of you wanted benchmarks so here's one comparing Shortfin with Nginx. The tests was performed with a 56.1 kB PNG image with keep-alive turned off on my laptop. The best result out of 3 tests is shown below.

tl;dr:

Shortfin: 18 914 req/sec

Nginx: 15 603 req/sec

SHORTFIN

  sudo ab -n 100 -c 100 http://127.0.0.1:40/timothy-johansson.png

  Server Software:        shortfin/0.9.5
  Server Hostname:        127.0.0.1
  Server Port:            40

  Document Path:          /timothy-johansson.png
  Document Length:        56089 bytes

  Concurrency Level:      100
  Time taken for tests:   0.005 seconds
  Complete requests:      100
  Failed requests:        0
  Write errors:           0
  Total transferred:      5618000 bytes
  HTML transferred:       5608900 bytes
  Requests per second:    18914.32 [#/sec] (mean)
  Time per request:       5.287 [ms] (mean)
  Time per request:       0.053 [ms] (mean, across all concurrent requests)
  Transfer rate:          1037701.56 [Kbytes/sec] received

  Connection Times (ms)
              min  mean[+/-sd] median   max
  Connect:        1    2   0.0      1       2
  Processing:     2    2   0.1      2       2
  Waiting:        1    1   0.2      1       2
  Total:          4    4   0.1      4       4
  ERROR: The median and mean for the initial connection time are more than twice the standard
       deviation apart. These results are NOT reliable.

  Percentage of the requests served within a certain time (ms)
  50%      4
  66%      4
  75%      4
  80%      4
  90%      4
  95%      4
  98%      4
  99%      4
  100%     4 (longest request)

NGINX

  sudo ab -n 100 -c 100 http://127.0.0.1:41/timothy-johansson.png

  Server Software:        nginx/1.2.6
  Server Hostname:        127.0.0.1
  Server Port:            41

  Document Path:          /timothy-johansson.png
  Document Length:        56089 bytes

  Concurrency Level:      100
  Time taken for tests:   0.006 seconds
  Complete requests:      100
  Failed requests:        0
  Write errors:           0
  Total transferred:      5631000 bytes
  HTML transferred:       5608900 bytes
  Requests per second:    15603.06 [#/sec] (mean)
  Time per request:       6.409 [ms] (mean)
  Time per request:       0.064 [ms] (mean, across all concurrent requests)
  Transfer rate:          858015.83 [Kbytes/sec] received

  Connection Times (ms)
              min  mean[+/-sd] median   max
  Connect:        1    2   0.7      2       3
  Processing:     1    2   0.5      1       3
  Waiting:        0    1   0.7      1       3
  Total:          2    4   0.9      4       5
  WARNING: The median and mean for the processing time are not within a normal deviation
        These results are probably not that reliable.

  Percentage of the requests served within a certain time (ms)
  50%      4
  66%      4
  75%      4
  80%      4
  90%      5
  95%      5
  98%      5
  99%      5
  100%     5 (longest request)

I don't know how valid a test of five milliseconds is to one of six milliseconds (much less one requesting a single known resource with no keep-alive). Though practically, even if those results held in a realistic test, is it still a viable alternative to nginx? You eliminate an enormous amount of flexibility (I admit -- I ♥ nginx) and proven trust for a margin-of-error theoretical speed advantage?

I understand the desire to get software out there, but webserver is a ridiculously hard nut to crack. nginx broke in through a new architectural paradigm.

Nope, I'm not trying to replace nginx. I love nginx too ;) The "challenge" was to build a very lightweight server that was faster than nginx - which I did. Then I've used it as a base for a various of projects (see my other comments).

Very cool. I've been doing the same thing lately, writing my own little web server that serves static files. It's been a fun learning experience!

Nice, in what language? I've really learned a lot from creating shortin! Shortfin is written in C so it's a nice project to spend some time on when I get tired of writing javascript and html ;)

I'm writing mine in C as well. I've never been too into web programming, but the lower-level stuff is quite interesting to me.

Keep it up!

Lots of people have asked "Why not Nginx? Why not thttpd?" ... but if all you want to host is static files, why not AWS S3 or similar CDN?

author should add a benchmark, comparison. or maybe it's just a fun, personal project and not ready for production.

I would recommend adding ipv6 support. Should be fairly easy to do in socket.{c,h} and in your configuration parser.

Waiting For Benchmarks.

Reply on news.ycombinator.com