BCHS software stack: BSD, C, httpd, SQLite
learnbchs.org
learnbchs.org
It's supposed to be simple, and it is simple compared to Apache, but it also has way fewer eyes on it. Ultimately, a big pile of string-handling C is likely to have some problems. There was a trivial-to-exploit server-crashing segfault in httpd's FastCGI implementation that was only fixed in the last month or so. There were also recent issues related to null bytes and line feeds in headers causing strange, exploitable misinterpretations of incoming requests in relayd. Much of this is now fixed, but I wouldn't be at all surprised if there were more low-hanging fruit remaining.
If you're looking for a lightweight HTTP server written in C, I recommend Lighttpd. It's older, more widely-used, and more standards-compliant. I'm not trying to dump on OpenBSD; I run it on my primary laptop. I just wouldn't use their HTTP tools for anything mission-critical yet.
lighttpd doesn't even chroot by default.
These security mechanisms are added and implemented for reasons and they're tested with the flame of time and non-hardened connections.
I'd take a battle-tested and mature implementation over a newer one any day, even if it's written in Malbolge.
As someone who has been dealing with Unix since the early 90s, most of those old C programs were always security nightmares. There were a few exceptions: djb's stuff, dovecot and (surprisingly) Apache. But most of the other popular C servers were absolutely riddled with buffer overflows and other security problems. The Morris worm, the entire rsh family, you name it. Sendmail was awful for a long time, too.
Memory corruption errors still make up 70% of security holes in modern C and C++ code, and that's after decades of improvement. That entire set of vulnerabilities could be avoided in the 90s by using Perl 5 or Java, which were insanely popular. (PHP came with its own supply of security holes, both in the interpreter, the language design, and the awful database APIs.)
We've had popular memory-safe options for decades. All Rust brings to the table is the ability to be memory safe without needing GC (which is great). But with rare exceptions, the popular C servers were always full of holes.
One key thing to keep in mind is that exploit mitigations are largely about limiting degrees of freedom. It's quite hard to contain an attacker who has gained code execution in a process (such as due to memory corruption) due to the massive amount of control they have. On the other hand, if you're able to prevent them from gaining such a strong foothold to begin with (such as by using safer languages which don't have such issues), you're in a much better spot because now they don't even gain any control over the process.
Like rust which downloads unverified crates from the internet. /s
It is an open source project with both commits and discussion happening in the open (tech@ in particular), so the barrier to understanding is reasonably acceptable. At some point it was also debated to move to nginx, but in the end httpd(8) was championed as a natural offshoot from relayd(8) which was already in base.
https://www.openbsdhandbook.com/services/webserver/nginx/#:~....
OpenBSD httpd would probably have never existed if nginx upstream reacted differently.
https://www.openbsd.org/papers/httpd-slides-asiabsdcon2015.p...
[0] The OpenBSD nginx port still includes the default chroot patch, ~8 years later, see https://raw.githubusercontent.com/sthen/nginx_chroot_patch/a... and https://github.com/openbsd/ports/blob/master/www/nginx/Makef...
I redid the whole project in a weekend in Perl and it was not only a better development experience but worked more reliably and was all around better. I miss CGI for developing web apps. It's straightforward and can easily (or at least in a straightforward way) be tested the same as a CLI tool.
A lot of the problems were my deficiencies in C and just general inexperience. I'm sure today there's a lot of nice CGI helper libraries that would make the process a lot easier. I also have a lot more experience in C so I'm confident I'd just do a better job writing such an app. I have no interest in it though. I'd rather "waste" cycles in a higher level language and solve problems in the domain rather than the tooling. But more power to the crazy diamonds writing web apps in C.
1) transistor density and memory cost never advanced past what they were in 2000
2) no one ever took the Linux kernel project seriously
Your second to last sentence validates this: Cycles are computing power, and power is cheap. It gets cheaper every day. The person who can write flawless C code, including the boilerplate, is expensive.
You need to make a product that is fast/efficient enough to do the job. It needs to be delivered on a schedule that is probably really stupid from an engineering point of view. Efficiency can be patched over by throwing extra resources at the problem (up to a point). Functionality and market timing can't be patched over in the same way.
I think it makes sense to throw resources at inefficient code to get yourself into a position where that efficiency has a meaningful consequence. Customers aren't buying your product/service based on the number of instructions or memory used to do a task.
It's a nice practice I guess
I recall imagining that what I was experiencing must have been similar to what early programmers submitting batch jobs went through. 0/10. Do not recommend.
The "web framework" might have changed - but it's not like that's the majority of Amazon.com
BCHS: OpenBSD, C, httpd and SQLite web stack - https://news.ycombinator.com/item?id=29988951 - Jan 2022 (149 comments)
BCHS stack – BSD, C, httpd, SQLite - https://news.ycombinator.com/item?id=28269399 - Aug 2021 (58 comments)
BCHS: The BSD, C, httpd, SQLite stack for the web - https://news.ycombinator.com/item?id=23148871 - May 2020 (1 comment)
OpenBSD, C, httpd and SQLite – Web App stack - https://news.ycombinator.com/item?id=17272225 - June 2018 (174 comments)
BCHS stack – BSD, C, httpd, SQLite - https://news.ycombinator.com/item?id=14580746 - June 2017 (75 comments)
Kwebapp: rapid BCHS web app development - https://news.ycombinator.com/item?id=14454381 - May 2017 (1 comment)
BCHS Stack - BSD, C, Httpd, SQLite - https://news.ycombinator.com/item?id=11763888 - May 2016 (68 comments)
mod_perl (https://perl.apache.org/) and mod_php (https://cwiki.apache.org/confluence/plugins/servlet/mobile?c...) helped to make Apache httpd (https://httpd.apache.org/) the number one web server in the early days of the web.
1. can you put a link to the running http file served by the example so we could see how the result looks?
2. Defining CGI would be helpful because I looked it up and found “computer generated imagery” which doesn’t seem like what you meant.
3. Might be good for this example to load the html from a file or SQLite to fully trivialize the whole stack, as this example doesn’t include the S in the acronym.
sometimes I think the Next.js examples folder at https://github.com/vercel/next.js/tree/canary/examples is just an amazing example of how best to market a software product to developers because it’s such a rich source of integrations, almost anyone can find a good starting point for a web app project in there, if BCHS had the 80:20 of examples ready to roll then maybe it could blow up, because BCHS a great idea to use the most battle tested solutions in existence! Keep it up! bravo!
The most primitive version is just launching one process per request, piping the HTTP request into stdin, and piping the response out of stdout.
It works, but you can imagine the startup latency is rough and it takes a lot of resources.
There are faster variations that try to reduce the overhead. Ironically FaaS is sort of a rebirth of CGI
Common Gateway Interface. It’s how we generated dynamic web content back in 1995.
Doing it with C is extremely painful.
[1]: https://en.m.wikipedia.org/wiki/Common_Gateway_Interface
How would it look any different than if it was served from a bash script or the world's most complicated .NET container in a Kubernetes instance?
This made me feel extremely old lol
This is the post with most comments (149)
Further you can read back previously added contents and make additional chagnes based on those. Like summing up table columns, or adding links to the first column values, or removing or fixing links based on authorization, to name a few.
Syntax aware template languages are a thing in the web space. They aren't the most popular but security people often like them because they lead to less mistakes. Some examples include soy, latte, and hack's XHP.
Found the old code on disk, snapshot here. The DOM parser, manipulator and free was 240 LOC.
httpd would be one of the things I'd look at, it's absolutely great software, but I wonder whether it really achieves the most simplicity from a developer point of view. For instance, in simpler deployments I would generally reach for Caddy [0], which does things such as certificate renewal automatically for me.
However, the part of the stack that really irks me is C. I'm a huge fan of C in an ideal world (where developers are perfect), and I respect the language for its role in the history of software development, and in the context of UNIX, however I just don't understand why use it in 2023 for something such as a web service. A web service is going to handle untrusted user input, deal with network boundaries, and is security-critical. A memory-unsafe language, where undefined behavior is easy to create but hard to find, which doesn't provide a lot of (useful) abstraction primitives other languages would provide, seems like the wrong choice. That's even before we start talking about how cumbersome it is to handle "strings" in C.
I'd wager Golang or Rust are always going to be better alternatives to C when it comes to developing web services. Golang makes deployment specially easy, while Rust provides similar or better performance than C, but provides more safety (memory and UB) and better abstraction primitives.
I believe I understand the purpose of this stack, and roughly who is going to enjoy it, just wondering whether I'm overlooking something, as I must admit I have never actually built a production service using CGI/C/httpd. I see this stack as something that's more philosophical rather than pragmatic towards development, if that makes sense, which is something I respect but wouldn't use (other than if I'm doing it just for fun).
Nor would you probably want to. In addition to the security nightmere that hooking an inexperienced c programmer's c program directly to the internet is, CGI is not really known for scaling all that well. Like if you were really doing this on a real high performance site you'd probably want to use FastCGI. But also you just wouldn't do this. If you want to be low level, at least use rust.
In 2023 people are using lambdas on was. Slow cgi is fast enough.
Single binary for distribution with assets/migrations embedded. Still need to build something substantial so I am sure there are edge cases/rough edges but so far it feels like a breath of fresh air compared to nodejs ecosystem.
Just writing a simple hello world in node/express downloads a gazillion dependencies and code that'll all be points of failure or mystery for lack of understanding. To understand them all to be able to write non trivial stuff is likely no different from doing httpd in c on Linux.
I've done stuff in go and it makes it lot easier to code.
> Is BCHS a joke?
> Software development is full of jokes. This is not one of them.> However, a server MAY omit header fields for which a value is determined only while generating the content.
I find omitting Transfer-Encoding quite understandable and reasonable; the whole purpose of HEAD is to say “don’t bother doing the work that GET would trigger, I don’t care about exactness, I’m just getting a general idea”. Though I do find cases where Content-Length is omitted, even on static resources, disappointing. Saw that happen for I think the first time a few weeks ago (that is, a Content-Length that was present in GET but absent in HEAD).
But certainly I’ve seen more than a few 405 Method Not Allowed responses to HEAD, which is definitely bad.
Don’t all the curly braces count as mustaches?
Today i would have considered mongoose(10k+ stars) which is also a mature c/c++ web server[1] if not the licence.
Also check out this thread(290+ pts) on tiny http servers in C. [2]
[1] https://github.com/cesanta/mongoose/tree/master/examples
rofl, exactly what i'm looking for in a web framework.
I honestly can't tell if this post is meant as satire or serious.
With all the memory-safety issues you can introduce by improperly using C, is this page meant to be taken sarcastically or are they really serious about this claim?
That said, the actual secret is to write simpler code. (And maybe use pledge+unveil, if your OS has it.)
C has the same tools, they however do not run with the compiler, but accomplish the same goal
Both, are still not immune to incompetence, the user is often the issue, hence the link