Comment by kragen | Hacker News Reader

Comment by kragen | Hacker News Reader

darkhttpd is cool, and it's pretty featureful for a small program, but it's also 2500 lines of code.

If you liked it, you might also like my server httpdito: http://canonical.org/~kragen/sw/dev3/server.s which has documentation in http://canonical.org/~kragen/sw/dev3/httpdito-readme. It's also an HTTP server in a single source file, but it's i386 assembly, for Linux, that doesn't use libc. It's only about 700 lines of code, and the executable is up to 2060 bytes now that I've added CSS and PDF support to it.

httpdito is, to my surprise, practically useful on occasion as an alternative to things like `python -m SimpleHTTPServer`.

Unlike the server in Koshkin's excellent comment https://news.ycombinator.com/item?id=26672683 httpdito does send CRLFs as the HTTP standard demands. Unfortunately it also requires them in the request. Also—and this is a key point on which most software falls down—httpdito's documentation contains a section entitled "Are you insane?", which really is a question every programmer ought to answer in their documentation.

It doesn't have most of the things darkhttpd has, though. It doesn't support IPv6, HEAD, directory listings, byte ranges, conditional requests, keepalive, name-based virtual hosting, sendfile, logging, chroot, or privsep, and doesn't run on Solaris, SPARC, or ARM. But if you're looking for features like "small memory footprint" and "no installation needed" and "no messing around with config files" it may actually be better than darkhttpd. Due in large part to improvements in modern Linux and its small memory footprint (normally 5 pages) it's efficient enough to saturate a gigabit connection.

It’s funny, this is the exact kind of thing I’d want on a non-x86 system. Raspberry Pico for example would be a great place for a 2KB HTTP server but with not support for HTTPS, IPv6, or for any non-x86 architectures it’s a bit of a non-starter for my use cases. Still, very cool project!

The CPU architecture is actually the least of your concerns there—I'm pretty sure qemu-user can run httpdito on ARM with less than an order of magnitude performance overhead. There are a lot of embedded systems where an HTTP transaction per second per MHz would be more than sufficient.

The bigger problem is that the Raspberry Pico is a dual-core Cortex-M0+, which doesn't have an MMU, so it can't run Linux and especially can't handle fork(). But httpdito is basically scripting the Linux system call interface in assembly language—it needs to run on top of a filesystem, an implementation of multitasking that provides allocation of different memory to different tasks, and a TCP/IP stack. Any one of these is probably a larger amount of complexity than the 296 CPU instructions in httpdito.

The smallest TCP/IP stack I know of is Adam Dunkels's uIP. Running `sloccount .` in uip/uip cloned from https://github.com/adamdunkels/uip gives a count of 2796 lines of source code ("generated using David A. Wheeler's 'SLOCCount'."). uIP can run successfully on systems with as little as 2KiB of RAM, as long as you have somewhere else to put the code, but for most uses lwIP is a better choice; it minimally needs 10KiB or so. uIP is part of Dunkels's Contiki, which includes a fairly full-featured web server and a somewhat less-full-featured browser. I think he got both the server and the browser to run in 16KiB of RAM on a Commodore PET, but not at the same time.

(twIP http://dunkels.com/adam/twip.html is only 139 bytes of C source but doesn't support TCP or any physical-layer protocol such as Ethernet, PPP,or SLIP.)

However, Adam Dunkels has also written Miniweb http://dunkels.com/adam/miniweb/, which implements HTTP and enough of TCP and IP to support it, in 400 lines of C. It needs at least 30 bytes of RAM. Like twIP, it doesn't provide a physical layer. But that's solvable.

Not having MMU means there's no virtual memory and instructions refer to physical memory addresses, cmiiw?

You say Linux won't work without MMU, it can't handle physical addresses? Moreover, why won't fork() work without MMU?

I would hazard a guess that fork is nowadays implemented by the clone(2) system call, which does use copy-on-write depending on its arguments.

It's possible that there are deep assumptions in the Linux code that some fundamental software or hardware memory handling capabilities are present.

Without an MMU, you can't do paging. That means fork() cannot do the normal copy-on-write business, because there's no page table to copy the entries in.

You also have no inter-process security, so everything can crash everything else including the kernel, and no swap.

It used to be the case that all Linux executables had to be loaded at the same virtual address, but ASLR may have removed requirements of this kind. https://stackoverflow.com/questions/38549972/why-elf-executa...

I'm pretty sure Linux ELF has always allowed you to specify the initial load address. When I first wrote StoneKnifeForth https://github.com/kragen/stoneknifeforth its load address was 0x1000, but at some point Linux stopped allowing load addresses lower than 0x10000 by default (vm.mmap_min_addr). I originally wrote it in 02008, using the lower load address, and fixed it in 02017. It's still not using 0x804800 like normal executables but 0x20000. ASLR does not affect this.

Maybe you mean that before ELF support, Linux a.out executables had to be loaded at a fixed virtual address? That's possible—I started using Linux daily in 01995, at which point a.out was already only supported for backward compatibility.

> Not having MMU means there's no virtual memory and instructions refer to physical memory addresses, cmiiw?

Pretty much, yeah.

> You say Linux won't work without MMU, it can't handle physical addresses? Moreover, why won't fork() work without MMU?

When httpdito fork()s two child processes, each of them starts receiving the HTTP request into the request buffer at `buf`. This works because the semantics of fork() give those two children two different buffers at the same memory address, one in each process's address space. The Linux userland relies relatively heavily on these semantics. It was a major obstacle to getting an SSH server running on cisco IOS, for example.

An event-driven server like darkhttpd is a much better fit for an MMUless system. Implementing multithreading is easy (it's half a page of assembly) but implementing memory mapping without an MMU requires some kind of interpreter.

(Actually you can implement fork() without virtual memory and without an MMU, for example with PDP-11-style segmentation, but the Cortex-M0+ doesn't have any of those facilities either.)

>"The Linux userland relies relatively heavily on these semantics. It was a major obstacle to getting an SSH server running on cisco IOS, for example."

Can you elaborate on this? Hasn't Cisco IOS at various times run on MIPS and X86 processors?

Unfortunately I'm just reporting secondhand rumors from people who worked at cisco, and I probably should have made that clear. So I don't know how IOS works at the machine-instruction level, just the command line.

The original Cisco IOS ran on 68000 series processors which lacked an MMU. Even the later 68K models used an "embedded" version of a processor which did not have an MMU. For example, the Cisco 2500 used a 680EC30. Regular 68030s had MMUs, but the "EC" model did not. Later versions did run on MIPS though.

Thank you!

You can build mainline linux without an MMU, and there are even pretty crazy setups where you can run it on a ARM cortex (though usually an M4). It is not a standard system though, very little software will run without modification. The biggest issue for such processors is usually actually lack of memory (they have relatively little built-in and most have no external memory busses. There's at least one project where the externel memory is bitbanged through gpio!).

Oh interesting! I didn't have any idea.

I think running Linux on a Cortex-A is entirely normal, FWIW.

Oh yeah, should have specified cortex-Ms (though the M7 is also reasonably linuxable).

For your website, it seems the files aren’t being encoded right because Firefox on iOS is rendering them as something else. I’m getting mojibake with apostrophes and the like:

> ### setup_socket: thereâ€™s little interesting to say here thatâ€™s not ### in, say, Beejâ€™s Socket Guide.

Thanks! Yeah, what happened was that Apache wasn't sending a content-type header until I tweaked the .htaccess file, which was about 14 minutes before you posted your comment. I had to ctrl-shift-r in Firefox to get it to notice the changed content-type.

Do you have plans for any of those features, or is it complete for the use cases it is solving?

No, I didn't write it to solve use cases at all, but rather because I thought it would be awesome to write a web server in assembly. As the readme says, we all have moments where we do things we regret. Youthful† lapses in judgment.

If you git clone http://canonical.org/~kragen/sw/dev3/.git you will see that I wrote most of it one weekend in 02013, removed the dependency on libc and added forking the next weekend, and tweaked it slightly over the next month and a half. Then in 02019 I added CSS and PDF to the mime-types list. So it's not, like, an ongoing project. It's just a program I wrote once, like most things in that directory.

I did just fix the documentation, though, because it was still claiming it was under 2000 bytes.

______

† Technically 37 is not actually that youthful.

Fair enough. Thanks for providing it, I have a few uses in mind.

You're welcome! I hope you enjoy it!

Hey why do you add an extra preceding zero on years?

https://en.wikipedia.org/wiki/Long_Now_Foundation

Nice. Have you considered publishing it under a FOSS license?

Thanks. I thought I'd done so when I wrote it in 02013! I've now corrected the omission by adding a Creative Commons Public Domain dedication. Thank you for pointing that out!

Reply on news.ycombinator.com