15 years later: remote code execution in qmail | Hacker News Reader

15 years later: remote code execution in qmail

qualys.com

259 pointsfanf2159 comments

Oh, dear. Another "will never happen" issue which, after not so long (in the history of mankind) takes place. And this one would not have been so complicated to fix (just enforce a limit as the author suggests).

    /* this line is unreachable */
    printf("You have reached unreachable code\n");

10 years later, it gets printed...

Why not exit/panic at that point? Seems like the right thing for any “unreachable” code.

Ego.

If you are really sure put __builtin_unreachable()

I'm pretty sure he didn't have that on most of the platforms he was targeting with his portable C code, it's not even there on some of them now.

what you do is you define a macro in a common header. Like UNREACHABLE().

Shoot I have done that in the past but it seems I actually wanted __builtin_trap().

Yeah, that footgun should really have a more explicit name.

__dear_compiler_i_promise_that_this_statement_is_unreachable_and_you_may_optimize_based_on_my_good_word_upon_penalty_of_undefined_behaviour()

I mean, what other semantics would anyone expect it to have—presuming they think for a moment about why it exists as its own thing in the first place?

Rust being Rust, the equivalent causes a safe teardown.

https://doc.rust-lang.org/std/macro.unreachable.html

The UB causing equivalent is of course unsafe.

https://doc.rust-lang.org/std/hint/fn.unreachable_unchecked....

Ok so what's the answer if you apply the question to __builtin_trap()?

Even better, just actually perform undefined behavior on that branch and the compiler will get rid of it.

This is literally what __builtin_unreachable() does. But it has the added benefit of the compiler understanding the intent so it works well with diagnostics.

But that's a builtin, rather than something that's part of the core language :)

Because it will never happen I guess. From my experience, it's because of this bad habbit of expecting the program to crash and provide a backtrace,the print is there for debugging. But you're right,it's not the sane way of doing it.

Didn't think of it? If I saw a line like that I'd be grateful that it printed at all.

I could see an argument (similar to for assert()) to omit such a check in most production builds. Maybe you could enable it for a small amount of hosts, so that if it does happen with any frequency, somebody sees it somewhere.

But not every "unthinkable" case being reached represents a security problem as it does in this case. So in the general case of how to approach such assert-type checks, an abort might take down the process when not necessary, turning it into a DoS.

Edit: I am not sure if my friend the downvoter realizes I am not trying to justify legit security bugs, just talking abstractly about varying approaches to handling asserts for unforeseen or "allegedly impossible" circumstances.

I didn't downvote you and I interpreted your comment the way you said you meant it. I do happen to disagree, though.

For a project in a space that's notoriously vulnerability prone - I mean, when it was released it was competing with Sendmail - it seems very reasonably cautious to pepper the code with panic handlers in "impossible" places. They won't slow the code down any because they should never be evaluated, but give nice, noisy explosions when unexpected things happen. There's no downside to doing this.

Solaris also notably bundles 32-bit binaries for most everything in /bin with 64-bit distributions, and that begins to look like a wise choice.

I recently observed that SmartOS does the same.

If we could tell a priori which unreachable cases were reachable it wouldn't be such a problem.

Asserting only in debug builds is rather ineffective because the unthinkable input isn't in your testsuite. If it were, then it wouldn't have been unthinkable.

> Asserting only in debug builds

Right, but I am also suggesting another approach: if you have a large fleet of machines or a client app with a lot of usage, you can enable it in a small percentage of runs and if it hits in the wild you still have a chance of learning about it, without tearing down the process for all users. I have seen that strategy be effective.

Very hard to know which behavior is appropriate in advance though. An assert that turns out to represent a benign circumstance that is already handled, multiplied by large numbers of machines, can be pretty annoying.

You have reached an imaginary program state. Please rotate your computer by 90 degrees and try again.

(paraphrased from a friend's answering-machine message)

I did that, but it's still a bit complex for me…

Where are you quoting this from?

That's quite interesting because the bug was there and was quite clear, and was not fixed by one of the most brilliant security persons out there (DJB). Maybe here the attitude really was the wrong one, considering how security oriented qmail was since the start, because the simplest possible fix does not seem to demand any other dimension, like complexity or performances.

While I think that he should have rewarded the original 2005 bug report I do not think that he did anything wrong after that. He was pretty clear about the memory limit for years now.

If the program only runs correctly with a memory limit in place, perhaps it should (if this is possible?) check for a memory limit when it starts and abort if it's missing.

Serious question: Was there a legitimate reason they didn't patch this stuff when it was first discovered?

It's not "they", it's Daniel J. Bernstein. That's the reason :)

(If you don't know: he is a top cryptographer that can amazingly correct code. However, he also has a very big ego...)

I just found this on Wikipedia where he raised his bug bounty and at the same time cited these "unexploitable" bugs

https://en.wikipedia.org/wiki/Qmail#Security_reward_and_Geor...

qmail was essentially unmaintained for a long time, people were distributing patches to it, but there were no upstream releases.

(Similar story with djbdns/tinydns.)

"Recently" he released his code with new licenses, so that people could finally start distributing updated versions, rather than the previous approach where lots of people were sharing conflicting patches for various features (e.g. IPv6 support for AAAA records in tinydns.)

notqmail.org is for long the defacto maintained version. They had this fixed long time ago.

I don't understand. Most people with a big ego would not want a critical vulnerability associated with them. Can you elaborate?

His ego has transcended objective reality and he claimed (in 2005, and continues to claim in 2020) it isn't a vulnerability.

he had such confidence in his software and abilities that he thought it was actually secure, and there were no bugs, and posted a bounty for any exploit that could be found. Patching it means acknowledging it's an exploit, and that his code was not without bugs.

Given that his principles of writing secure software (included in the Qmail guarantee[1]) includes this: "7. Write bug-free code." that might be a bit hard for him to swallow.

1: https://cr.yp.to/qmail/guarantee.html

Well, he used it with memory limits command line switches, so it could never be exploited. So he was technically correct. One should not use so much memory for a mail server, way too risky.

Problem is, these switches were not default, people didnt use it because they are dumb, and DJB never cared to properly maintain it. like limiting memory per default, 32bit only builds or such.

> Problem is, these switches were not default, people didnt use it because they are dumb

Pushing complexity from a very small group (in this case, one person) who knows the system intimately to many orders of magnitude more people that are meant to have a functional knowledge of how it operates but not necessarily be intimate with it is a losing proposition, and not any tenet of how I would consider developing secure software.

If the software is only supposed to be run under process limits, and over a specific process limit all bets are off security wise, then the program should probably check and report problems with large process limits when it starts. Or, as you posit, dying if built for 64 bit, since its assumptions don't necessarily hold.

My opinion on this is that if you're going to claim that you write the most secure software in the world, it should be secure by default. It shouldn't require you to modify the configuration in a particular way, or start it in a particular way, in order to be secure. The more details you need to know in order to secure something, more less likely you'll tick off all those boxes.

To me, this is just DJB's ego not allowing him to admit that he made mistakes.

> One should not use so much memory for a mail server, way too risky.

Is there a table or formula I can consult that will give this particular dumb person(myself) a handy guide for what amounts of addressable memory will introduce security risks for particular applications? Apparently more than 32-bits is obviously[0] a problem for email; what about databases? Should I feel bad I use more than 64GB of memory in my DB installations? Am I being irresponsible? What about web servers? How much risk does each additional bit of memory add?

My final question is, why does pretty much every other software maintainer not have a problem fixing the memory allocation themselves, obviating the need for external tools to fix these issues? I guess they're going the extra mile!

[0] So obvious a problem that sendmail, postfix, and exim don't require me to apply workarounds for it for some reason. Very irresponsible of them, if you ask me.

He did reward some other bounties that he made, I can only think the ones from his cryptographic algorithms at the moment though.

Edit: See https://news.ycombinator.com/item?id=23250748

He asserted that it was unexploitable, and thus not a vulnerability. Fixing the code would have implied that it might have been a vulnerability, and he might have been wrong. Can't have that.

http://cr.yp.to/talks/2007.11.02/slides.pdf I don't see ego in these slides. I see a brilliant programmer acknowledging his mistakes and learning from them. I really enjoyed running qmail in early 2000s and following djb's crypto work later. He is brilliant indeed.

You should read some of his public.... ekhm... “discussions” with Wietse Venema on various security forums in the 90’s. It was very entertaining, but also clearly showcasing djb’s huge ego.

A lot of it would get you banned from HN and similar fora today.

Here is my favourite that I remember to this day: https://seclists.org/bugtraq/1998/Nov/117

Hah, "How many people relaxed after installing tcpd -DPARANOID, instead of pestering their vendors for a real fix?" Sounds similar here ;)

Yeah, the lack of self-reflection is pretty palpable.

The funny thing about that is I find his code to be very difficult to read (even just the snippets in the linked CVE illustrate this).

And his attitude is just bonkers to me. "I'm not going to fix this exploitable security issue because I assume that people will configure their environment in a particular way." What? That's... flat-out irresponsible.

He does not have any responsibility against anyone. He released his software in public domain with the source included for free.

Putting aside question of if can have responsibility for freely-released work (especially when one has made a big deal of money offered in exchange for this kind of finding), at the time this bug was discovered the software was emphatically not in the public domain and difficult to distribute modified versions of despite available source.

When you release software for the world to use, tell everyone it's secure, even put up a bug bounty… that kinda means you are taking responsibility.

https://twitter.com/FiloSottile/status/1262854396934791168

Filo is upset that he did not bother to check the code (that originally came from SUPERCOP, a benchmarking tool) he blindly included in go's xcrypto. Here is DJB's tweet: https://twitter.com/hashbreaker/status/1108637226089496577

(regardless, you should not directly encrypt a large amount of data, even nacl suggest against it https://nacl.cr.yp.to/valid.html)

In addition both Filo and Garrett have a bone to pick with DJB due to their personal political beliefs and his involvement in the Appelbaum case and I found both of them to be extremely dislikeable and unable to accept their own faults in personal discussions that I had with them in the past (regarding different issues). Considering that this was a subpost my opinion of them is even lower now.

Your criticism of the messenger of further evidence of djb's longstanding refusal to deal straightforwardly with security reports is not on topic, IMO.

Not everything is a simple dichotomy.

(regarding qmail) It was a security bug back in 2005. It stopped being a security bug when DJB mentioned on the official page about the memory limits.

Regarding the salsa20 implementation: I just mentioned in my previous message why this was not a bug and the only reason that people were upset over it was due to Filo's incompetence.

As for evidence of DJB dealing straightforwardly with security reports: https://news.ycombinator.com/item?id=23250748

I think that https://old.reddit.com/r/crypto/comments/72w42c/statement_re... would be a better example of DJB not properly handling security reports.

salsa20 was added in 2012, the warning file was added into the repository in 2016 the earliest (it is not clear when--which is vary bad for security and also shows the move was not advertised.)

Incompetence is a strong word on the wrong target...

> the warning file was added into the repository in 2016 the earliest

There is no official public repository. It gets released in tarballs.

This does indeed make him seem slightly less incompetent though.

> which is vary bad for security

It is a framework for benchmarking cryptographic algorithms.

My opinions around DJB have very little to do with my political beliefs, but rather more to do with spending time with people who are far better at cryptography than he is without having anything approaching his ego.

> even nacl suggest against it

You're defending djb's decision by pointing to another of his projects, which in turn cites another email from djb. I'm not saying you're wrong¸ but it's not exactly a reviewed position.

I meant it in another way. If he had taken DJB's advice this would not be an issue, so DJB can't be blamed for it. Sorry for the misunderstanding.

That being said, he seems to agree with this. After all he followed said advice in his tool age.

> a bone to pick with DJB due to their personal political beliefs

"Political beliefs" is a weird way to say DJB has stepped up to defend at least three people accused of sexual abuse by multiple victims.

I actually had https://blog.cr.yp.to/20160607-dueprocess.html in mind when mentioning political beliefs, but even then I think that this still falls under the "political belief" label - the belief that anyone accused of rape should not have any form of defence. Anyway, the person who is being sued was part of a harassment campaign against multiple people, including Bernstein himself, so I can't see why it would be a bad thing for him to send his declaration.

For anyone interested, his declaration is here: https://www.courtlistener.com/recap/gov.uscourts.cand.340308...

And here is the most objective and complete story of the Appelbaum events that I have personally seen so far https://github.com/Enegnei/JacobAppelbaumLeavesTor/blob/mast... (if anyone has anything better please do let me know)

> accused of sexual abuse by multiple victims

Including false accusations by others in the name of the so called victims. Such as the "Alice" case. (if I am not mistaken this specific accusation was published by the person being sued themselves)

People can be onboard 100% with his argument for due process (I am) and simultaneously 100% against his association with Jake Appelbaum, whose (perhaps subcriminal) misbehavior was widely reported in private in our industry prior to the bevy of as-yet-unproven rape accusations.

Appelbaum is and has been a scumbag regardless of the fact of whether or not he has been adjudicated a rapist in a court of law.

People who associate with scumbags (and, indeed, defend them in particular) aren’t great, and can and should be subject to criticism for their choices regarding scumbags.

Fortunately, it isn’t a simple dichotomy. I agree with due process for imprisoning people. I also agree in public criticism of entirely legal misbehavior and freedom of association. I don’t respect people who defend scumbags socially (defending subcriminal scumbags from prison is another matter), and djb is certainly that.

> Appelbaum is and has been a scumbag

I do not know him, do you? Most accusations against him that I have seen have been either by the person being sued or some form of hearsay.

It is not too unlikely that he is a scumbag to be honest, but it is still something that I do not know.

> and, indeed, defend them in particular

I contest this claim. He did not defend Appelbaum in this instance, in his declaration even he claims that he is unaware whether Appelbaum is a rapist. The lawsuit is against Lovecruft specifically. Regardless, I do not believe that scumbags do not deserve to be defended. Everyone does, as long as the defence has reasonable points that is.

Btw, can't this post of yours be interpreted as defending Lovecruft if we follow this logic? If so I find this ironic that you try to criticize Bernstein of something that you are doing yourself.

> I also agree in public criticism of entirely legal misbehavior

You must love sites like Kiwifarms then. It is one thing to have open criticism and debate and another to have dog-piling and harassment based on roumors - the ability to defend yourself and have others defend you is one of the most important things that distinguishes the two.

> and freedom of association

Do you also believe that people should be free to refuse to deal with minorities by any chance? This is something that is implied by the freedom of association after all.

> I don’t respect people who defend scumbags socially

Again, the pot calling the kettle black. I do not get this logic to be honest, I will explain why with an example. Let's take a scambag, Jeff Bezos for example, and I start saying that he is a murderer out of nowhere. Is nobody allowed to defend him or ask for evidence just because he is a scambag?

The original qMail author, Dan Berstein (DJB) was so convinced of the infallibility of his code that he put up a monetary reward for any exploits. This context came about because DJB, as a professor of Computer Science, maintained that it is completely within the realm of reality to write unbuggy, yet complicated and highly functional code.

DJB welched on every claim at that bounty, and refused to pay out. He is an egotistical over-selfconfident person. He wrote good code back in the day, but he lost the forest for the trees.

You're being downvoted, but from what I can tell you're right. DJB is undoubtedly a brilliant computer scientist, but it seems he'd be very quick to tell you that. I see this as a loss to our profession because I think that DJB-minus-ego could make even more awesome things.

But that DJB doesn't exist. I can come up with much bigger losses if they don't have to exist.

I presume that a DJB-minus-ego wouldn't have had any motivation to become DJB in the first place ;)

Oh, I don't know about that. There are lots of brilliant computer scientists and mathematicians with reputations for also being really nice and humble people. The smartest people I know freely admit when they're at the edge of their knowledge and abilities, and are the first to tell you when they've made mistakes.

You're talking about other people's motivations. But DJB minus ego wouldn't be some other person, he'd still be DJB; just "DJB if you cut a certain part of his brain out." And I don't think that that person would have any other motivation-generating part of his brain left. His works specifically seem entirely ego-driven. DJB minus the ego probably wouldn't have had any interest in becoming an information-worker at all. (I'd like to think he'd become a janitor, like the protagonist of Flowers for Algernon. :P)

I concede this point. May I modify my original statement? I wish DJB took motivation from non-ego sources, so that we could have the wonderful things he’s made, but even more wonderful because they were made by someone who knew they were fallible.

Almost everyone who has worked really hard to be great at something has an ego about it. In most cases, they really wanted to be better than (often some specific) others.

To pick some well know people, I'd say Feynman and Einstein both had massive egos. They "knew" they were smart. They also had reputations for being really nice and humble.

It would be simpler to just admit that you think DJB is a bit of a jerk. Linus Torvalds is a bit of a jerk too, in most people's estimation. Brilliant jerks, we call them.

They can make great workers and are always terrible leaders.

There’s a big difference between self-assessed qualifications, and ability to admit fallibility. Good scientists know they know a lot. Great scientists know that they don’t know a lot, and admitting that you might be wrong is critical for the scientific method to work.

So DJB is brilliant, but if he admitted that he could make mistakes (or even that a compiled could mis-compile his flawless code), then he might have put in failsafes like unreachable code assertions that would have meant that we wouldn’t be discussing this today.

I don’t think he’s a jerk. I don’t know enough about him; maybe he’s the nicest, kindest guy around. I do think the evidence suggests that he’s arrogant, though, and that’s not a good look on anyone.

So wide-spread is his ego, that it reminds me of this quote from Donnie J. Barnes (of RedHat): "I went on IRC once. I was mistaken for Dan Bernstein. I still have nightmares."

DJB paid out $1,000 in 2009 to Matthew Dempsky for a djbdns security vulnerability.

https://marc.info/?l=djbdns&m=123613000920446&w=2

I used to work with Matthew. He's next level smart, and very modest. We are very lucky he is a good hacker and not an evil hacker. :-)

> The next release of djbdns will be backed by a new security guarantee.

I was a bit curious about this wording which seems quite weasel-wordy at first glance, so these are the snapshots of the page that have been saved by Archive here https://archive.vn/https://cr.yp.to/djbdns/guarantee.html

I did a rough diff of the two first saved snapshots (which span 2009), and got this:

    @@ -6,7 +6,7 @@
     
     The djbdns security guarantee
     
    -I offer $500 to the first person to publicly report a verifiable security hole in the latest version of djbdns.
    +I offer $1000 to the first person to publicly report a verifiable security hole in the latest version of djbdns.
     
    @@ -14,17 +14,25 @@
    -Bugs outside of djbdns, such as OS bugs or browser bugs, do not qualify. The vulnerability of DNS to forgery does not qualify. Denial-of-service attacks do not qualify. (An attacker can easily take down the Domain Name System, or selected parts of it; this is not news.)
    +Examples of problems that do not qualify:
    +
    +
    +
    +    Bugs outside of djbdns, such as OS bugs or browser bugs. (People could seize control of BIND 9.1 through an OpenSSL buffer overflow, but that was a bug in OpenSSL, not in BIND.)
    +
    +    The vulnerability of DNS to forgery. (BIND's port reuse makes blind forgery much less expensive, but this is a quantitative difference, not a qualitative difference. The DNS architecture needs cryptographic protection.)
    +
    +    Denial-of-service attacks. (BIND 9's fragility makes denial of service completely trivial; but an attacker can easily take down the Domain Name System without using any of BIND's bugs. The DNS architecture needs to be decentralized.)

I don't think this looks unreasonable in terms of actual consequences/definitions, but it is interesting how much effort and verbiage he spends on pointing out the flaws of other DNS servers.

Can you give an example of a claim he "welched" on? Google isn't pointing me to anything except the fact that email is unencrypted which I tend to agree is out of scope.

As discussed in TFA, he didn't pay out for Georgi Guninski's original discovery in 2005 nor Qualys' rediscovery + working RCE in 2020.

This is great. I remember when this came out. Georgi Guninski was a very accomplished bug hunter. Well known for finding vulnerabilities in web browers (ie. internet explorer) But the security community frowned upon that a bit as "not real hacking" so he was like "hold my beer", went after the hardest "real" target you could find and this was the result.

The guy is a legend

The surprising thing here for me was that you can send qmail a 4GB email message and it will not immediately reject it in the default configuration.

4GB was impossible when qmail was last updated.

According to Wikipedia DJB's last release was 1998. 64-bit Alpha and SPARC V9 were in widespread use by then. Even Linux was ported to Alpha in 1995.[1] See https://www.linuxjournal.com/article/1044

[1] Actually, Jim Paradis announced the Linux/Alpha development kit January 23, 1995 on comp.os.linux.announce (Message-ID: 3g0787$59i@kruuna.Helsinki.FI). So it's possible he had the kernel ported in late 1994. He announced a fully functional distribution July 1, 1995. (Message-ID: 3t34ph$mi@kruuna.helsinki.fi)

It's been years (2009) since I administered a mail server but I think this is also true of Sendmail.

"Finally, we also discovered two minor vulnerabilities in qmail-verify (a third-party qmail patch that is included in, for example, Debian's qmail package): CVE-2020-3811 (a mail-address verification bypass), and CVE-2020-3812 (a local information disclosure)."

The vulnerabilities were introduced in a third party patch, not source code that was authored or advocated by djb.

"As recommended by Daniel J. Bernstein, qmail can be protected against all three 2005 CVEs by placing a low, configurable memory limit (a "softlimit") in the startup scripts of all qmail services."

Seems much simpler than patching, let alone trusting someone else's patches.

Interesting that they developed their exploit against Debian's default configuration. qmail from its source at https://cr.yp.to/qmail.html comes with no such default configuration.

"About our new discovery, Daniel J. Bernstein issues the following statement:

""https://cr.yp.to/qmail/guarantee.html has for many years mentioned qmail's assumption that allocated array lengths fit comfortably into 32 bits. I run each qmail service under softlimit -m12345678, and I recommend the same for other installations.""

> The vulnerabilities were introduced in a third party patch

Which patch are you talking about? I don't see anything about a patch in the CVE.

> Seems much simpler than patching, let alone trusting someone else's patches.

We can patch the issue for good (and with a simple patch, apparently), or we can rely on users not to mess with seemingly unrelated init scripts. Do you trust third party users more than a (likely heavily scrutinised) small third party patch?

qmail-verify. It is an "update" of the third party "realrcptto" patch by another third party. It says "third party patch" right there in the sentence I quoted. There is no qmail-verify in original qmail.

As to the second question, I trust softlimit from daemontools and that is what I use. https://cr.yp.to/daemontools/softlimit.html

Not sure what "messing with seemingly unrelated init scripts" means. Can you be more specific?

I could trust softlimit all right, but my point was that I wouldn't trust users to wield it correctly.

softlimit is not difficult to use, certainly no more difficult than patch. programs like tinydns-conf for example autmatically generate a run script containing softlimit with a recommended -m setting. maybe qmail needs something similar.

Correction: With tinydns, it's a -d setting, not an -m setting

I was talking about patching upstream, okay? It's not like the patch would have to be individually applied to each installation by its own admin. Patch upstream once, and poof, all users worldwide are safe as soon as they update.

Perhaps "Seems much simpler than patching" was an ambiguous statement. I was only referring to the perspective I take as an admin for one user: me. I already use softlimit on a daily basis so it is simpler for me than patching. To me, it seems like a cleaner solution. The two complete mitigations presented were a choice between (a) using softlimit and the -m setting used by qmail's author versus (b) applying a third party patch to fix problems caused by another third party patch (qmail-verify, which was an "upadte" of yet another third party patch) and third party configuration settings (Debian). I thought that was noteworthy.

Do we know if they have applied for the $500 bug bounty from DJB?

My guess is that he'd never accept it. He originally didn't allow people to distribute qmail with changes; this is squarely a Debian issue.

What did Debian change to introduce this vulnerability?

Nothing, per se. But they don't set up softlimits or message size limits by default, both of which you should definitely do... it's just that the values you put in are specific to your situation.

If you have DJB's ego yelling at you that the default configuration is safe, why would you bother setting limits to fix it?

Because you care more about the users than about DJB's ego.

Then you should be running Postfix.

DJB has been telling people to run it with the memory limits since 2005.

If he takes that stance than it will cast doubt on everything else he has done. If there is a footgun on the wall, by the end it must go off, and chaining "non-exploitable" bugs together is routine stuff for hackers.

the grown-ups moved to postfix a long time ago

It's been 20 years; I don't think he's concerned.

Ahhh I see now that "qmail last had an official release in the late 90's". That changes everything, obviously.

In case it wasn't sarcasm: I would say it does not. What would, would be a note on the website explicitly mentioning that Qmail is unmaintained, and people should either take over maintenance or use something else.

Seeing no such disclaimer at https://cr.yp.to/qmail.html means DJB is (possibly unintentionally) still vouching for it. Right now in 2020.

When you say "this is squarely a Debian issue", do you mean "in his opinion", or that you agree?

Yeah, it was denied.

About our new discovery, Daniel J. Bernstein issues the following statement:

    "https://cr.yp.to/qmail/guarantee.html has for many years mentioned
    qmail's assumption that allocated array lengths fit comfortably into
    32 bits. I run each qmail service under softlimit -m12345678, and I
    recommend the same for other installations."

And from his "guarantee"

In May 2005, Georgi Guninski claimed that some potential 64-bit portability problems allowed a ``remote exploit in qmail-smtpd.'' This claim is denied. Nobody gives gigabytes of memory to each qmail-smtpd process, so there is no problem with qmail's assumption that allocated array lengths fit comfortably into 32 bits.

Under mitigations they list:

As recommended by Daniel J. Bernstein, qmail can be protected against all three 2005 CVEs by placing a low, configurable memory limit (a "softlimit") in the startup scripts of all qmail services.

Alternatively:

qmail can be protected against the RCE (Remote Code Execution) by configuring the file "control/databytes", which contains the maximum size of a mail message (this file does not exist by default, and qmail is therefore remotely exploitable in its default configuration).

Unfortunately, this does not protect qmail against the LPE (Local Privilege Escalation), because the file "control/databytes" is used exclusively by qmail-smtpd.

Alternatively:

- an updated version of qmail-verify will be available at https://free.acrconsulting.co.uk/email/qmail-verify.html after the Coordinated Release Date;

- the developers of notqmail (https://notqmail.org/) have written their own patches for the three 2005 CVEs and have started to systematically fix all integer overflows and signedness errors in qmail.

So the app is vulnerable by default, yet the author is claiming this doesn't matter, because he instructs how to run it in a safe way?

Correct, or am I oversimplifying/missing something?

Maybe add a reference to how ElasticSearch (or some other new-pop tech) catches heck for the same?

I think you're being downvoted because other things being insecure is not relevant or an excuse.

I'll reply to myself because I cannot edit now.

I meant: my parent comment (@allover) was missing a reference to how ES is insecure by default, this community gives them heck (rightly so) and that this comparison (qmail v. ES) could have been added (ie: was missing) from his post.

For a result of: this is a qmail bug that could/should be fixed AND ES should fix theirs too.

I'm for sure (I thought obviously) not excusing either qmail or ES from being insecure by default or for their "fix" to be: "you're doing it wrong".

I don't think my karma will ever recover from this (Tiger King joke)

qmail last had an official release in the late 90's. everything else is third-party forks / patches. Back in the day, I upgraded many systems from sendmail to qmail. However, that was a very long time ago... It's been over 15 years since I've done something like that.

Nowadays, the author should be telling people to install postfix.

Correct.

The app is vulnerable if it runs in an unsafe environment that allows qmail to access more than 4GB (an absurdly large value when qmail was published in 1997 -- it would cost $5000 plus a rare, expensive machine to hold it).

djb's view is that the environment is the responsibility of the admin, not the program's responsibility to enforce sane defaults. This is of course debatable.

If the admin uses a recommended environment (low memory limit), there is no exploitability.

So it's not secure by default.

It is in the sense that the author did specify AFAIK the bounds within it should run, no?

Where and when? If I had installed qmail on a 64 bit machine in 2004, how would I have known that it was remotely exploitable?

    -m12345678

Ah, magic numbers.

DJB makes great code, but it always leaves a sour taste to use his code, because he has such a big ego and refuses to admit when he makes a mistake. He can't admit that it's possible to be the best but still make mistakes.

I really hate supporting that.

I once was talking to one of the household-name DNS guys and asked what he thought about djbdns and he said something along the lines of: "It's unfortunate that he's so abrasive, because he has some really great ideas that nobody will listen to because of it."

I've more recently come a little bit to terms with this whole abrasive thing being a brain chemistry artifact, and not just being an asshole for assholes sake. Because it seems to disproportionately impact those of us in the technical field. Not to excuse it, but to understand it.

> Not to excuse it, but to understand it.

Indeed. I don’t have to abide such people, and neither do you. Now if he had found a cure for cancer, would we have more tolerance?

I'm not a psychologist so I should probably shut up, but I'll continue:

It seems like it does hit the tech field more often, yet overwhelmingly I see leading people in tech get softer, more empathetic, and more open to outside ideas the longer they work and the more experience they gain.

Perhaps this is selection bias (the people I see are the "popular" ones, so of course they are the ones with more social awareness), or perhaps there is no hard and fast "genius requires ego". Note: I'm aware you didn't say that it did, you only mentioned that abrasive-ness seems disproportionately frequent in the tech field.

I'm all for improving our understanding and ways of communication - I've definitely known people in the "asshole" category that were surprised and bothered to discover that was how they came across - but I'm not interested in normalizing abuse as a necessary cost. Which, again, you didn't say nor imply, but the concepts are related to what you've brought up, so I mention it here because the topic is interesting, not as a counterpoint.

LPT: take a shortcuts to success by listening to djb and plagiarzing his ideas.

Obvious Philosophy of Mind question of the day: What is the difference between a brain chemistry artifact and being an asshole for assholes sake?

> I really hate supporting that.

It appears the only victim of your spite is yourself. (Hmmm... there's an old saying along these lines.)

> and refuses to admit

I wish people stopped repeating it over and over. See https://news.ycombinator.com/item?id=23250748

So he paid a bonus one time? And that negates all the other times he refused to admit he was wrong (including the time this very article is about, twice)?

> So he paid a bonus one time?

Yes, which invalidates your statement that said "refuses to admit when he makes a mistake".

> And that negates all the other times he refused to admit he was wrong

No, I personally think that he should have awarded the 2005 report.

Considering that we have one example where he rewarded the bountry and one where he did not I would say that "he has such a big ego and refuses to admit when he makes a mistake" is just as valid as "he is humble and admits it when he makes a mistake"

Maybe this will be useful to someone adding the following flags LDFLAGS='-z relro -z now' help with one of the holes (they mentioned it, but not what flags) Check out: https://github.com/slimm609/checksec.sh

  * System-wide ASLR (kernel.randomize_va_space): Full (Setting: 2)
  * Does the CPU support NX: Yes
  * Core-Dumps access to all users: Restricted
           COMMAND         PID RELRO      STACK CANARY            SECCOMP          NX/PaX        PIE                     FORTIFY
  # checksec --proc-all |grep qmail
      qmail-injectXXXXX27 Full RELRO      Canary found            Seccomp-bpf      NX enabled    PIE enabled             Yes
       qmail-queueXXXXX97 Full RELRO      Canary found            Seccomp-bpf      NX enabled    PIE enabled             Yes
        qmail-sendXXXXX78 Full RELRO      Canary found            Seccomp-bpf      NX enabled    PIE enabled             Yes
      qmail-lspawnXXXXX89 Full RELRO      Canary found            Seccomp-bpf      NX enabled    PIE enabled             Yes
      qmail-rspawnXXXXX90 Full RELRO      Canary found            Seccomp-bpf      NX enabled    PIE enabled             Yes
       qmail-cleanXXXXX91 Full RELRO      Canary found            Seccomp-bpf      NX enabled    PIE enabled             Yes

This is why you don't run software that fossilized back in the nineties. Qmail was solid code but hasn't received the support and on-going development it needed. Use postfix instead for a secure MTA.

Your prejudice against out-of-fashion software is unfounded. Qmail had a bug because qmail had a bug, not because it hasn't been rewritten in the past week. The major web browsers, for example, are all under very active development, yet new RCE (!) bugs are regularly introduced into them.

The bug was known 15 years back, and not fixed.

"But we need to rewrite it with Rust!!" Crazy..

I said nothing about rust, and the alternative I proposed (postfix) is written in C.

This isn't "out-of-fasion prejudice". This is a bug not being fixed for over a decade, which is clear evidence that it's not being adequately maintained. I've got nothing against qmail in principle, but it clearly hasn't received the attention it needs.

Integer overflow bug causing manual memory allocation bug leading to RCE. All too common. We need to stop relying on software written in unsafe languages.

It is about safe and unsafe implementations rather than safe and unsafe languages.

This specific type of vulnerability and the tens of thousands of similar ones like it would absolutely have been prevented by a safe programming language.

This specific type of vulnerability would not be an issue in a safe C implementation for example, just like it would be an issue in an unsafe rust implementation.

"safe C" and "unsafe Rust" sound like variant languages to me.

How is "safe C" a variant language when it is within what the standard allows?

The unsafeness of C is pretty widespread and fundamental. Whatever code you end up with is either going to be incompatible with normal compilers, or it's going to be so full of library calls that you can barely recognize it as C anymore.

Unless you count sandboxing the entire program, and allowing errors inside the sandbox as long as they don't escape. Then C-to-WASM would be a safe C compiler, I guess.

> Whatever code you end up with is either going to be incompatible with normal compilers

I do not believe that this is necessarily the case.

There have been research projects that have attempted to make a standards compliant "safe C" a number of different times. You either end up with a 3x execution time overhead, only support a subset, or break programs that do unsafe type punning. As a result, this is not viable.

Can we name any programmers who are actually capable of a safe implementation using an unsafe language? DJB failed, Knuth's code has bugs, how can I expect you or I to succeed?

I am not talking about safe implementations of programs but rather safe implementations of languages. A (kind of bad example because it does not catch everything) is the address sanitiser that clang and gcc have.

That's a good point, but on the other hand, who wrote the languages? Is it possible for a human to write a language which another human can use and create no bugs?

In either case - unsafe language and safe implementation, safe language and unsafe implementation - there is the possibility for bugs, and definitely security ones. Just because you can't cause integer overflow doesn't mean you can't exfiltrate petabytes of data.

I think the major security consideration should be a comprehensive review of the entire system. Regardless of the language or implementation, a regular architectural and non-functional review of all the components in a systematic way is going to provide more security gains than any other single thing. Maybe we should be focusing less on the tools we use and more on improving that workflow.

Postfix has 9 CVEs. Qmail has 1. It would seem the numbers don't agree with you.

Postfix has much more functionality out of the box so the comparison isn't really fair.

Both are examples of extremely well designed software written with care by security minded persons.

> Postfix has much more functionality out of the box so the comparison isn't really fair.

But that's kind of the point. 20 years worth of feature accumulation is 20 years worth of additional attack surface, which is a liability if all you need is what qmail does.

Does anybody still use qmail in Production. I see on DJB's website that it is used in 700K sites but I doubt that assumption. Also the code has not been updated in almost a decade. How or why do sysadmins decide to go with qmail in 2020

Had a friend of mine just unrack an apple server preparing for a data center move.

You can use a heap of patches for qmail, but I wouldn't recommend it.

I was using Courier MTA for a while for self-hosting email.

I may do so again, and will likely use mailcow or mail in a box. Neither are based on qmail.

I would bet a lot of upvotes on this come from people misreading this as Gmail as I did :)

No. I ran several Qmails for many years on Mandrake with daemontools to supervise. All compiled from source. Lovely. "Life with Qmail" is a classic.

I needed some additional flexibility and switched to Exim for my MTA of choice instead but have fond memories of Qmail.

One thing that might help to improve overall qmail code quality (even though it's already written by a genius of our time) is to rewrite qmail in Rust[1].

[1] https://github.com/notqmail/notqmail/issues/115

The only way that could improve the quality is for DJB to rewrite it in Rust. If anyone else rewrote it in Rust, it would just contain different security bugs, and probably run worse.

The code is concise k&r c, converting it to any language could be an informative experience

Reply on news.ycombinator.com