Hacker News Reader

Comment by _urga | Hacker News Reader

  Size. A payload of one million characters should probably
  be rejected without further analysis. As well as checking
  the total size, it is good to check the sizes of the parts.

Another thing to check, that is often overlooked, is Quantity.

Every loop should have a limit. There should be no unbounded object allocation. Usually, it's not the big things that get you but the sheer number of small things.

For example, a 20 MB email with 4 million empty attachments:

https://snyk.io/blog/how-to-crash-an-email-server-with-a-sin...

Further examples that affected ClamAV and SpamAssassin:

https://blog.clamav.net/2019/11/clamav-01021-and-01015-patch...

http://mail-archives.apache.org/mod_mbox/spamassassin-announ...

View on HN

The only problem with this approach that I have encountered is that is requires defining fairly arbitrary limits that are either so small that some user will inevitably run into them or so large that you're not getting as much protection as you think.

NB I don't know what the answer to this is, but pretty much any time a system I have been involved with contains a "reasonable" limit then people want more than that pretty quickly!

> The only problem with this approach that I have encountered is that is requires defining fairly arbitrary limits that are either so small that some user will inevitably run into them or so large that you're not getting as much protection as you think.

Would an answer be to have limits that are tunable? I.e., something like "Attachment limit exceeded. Enter new temporary attachment limit for re-scanning: _____"?

The problem you describe is not actually specific to this approach, the same would apply to any Size check, or any limit in general.

In fact, I think the problem is less relevant to a Quantity check, where an order of magnitude headroom above normal usage goes a long way, more so than a Size check.

For example, do you know of people who receive 10,000 attachments per email? This limit would be far and away above any reasonable usage and yet provide decent protection at the same time.

It's essentially the arbitrariness of that 10,000 number that I'm queasy about.

e.g. When I thought of a "reasonable" maximum number of attachments on an email I'd probably say 64.

It can be a random number between 64 and 10000, inclusive, the arbitrariness of it is the point: there is a bound; the (loop/size/whatever) is not unbounded.

But that's just from the 'security' perspective - if it is a low number then users are likely to hit it and complain, and for bug tracing I suspect everyone would want the number to be constant.

"When I thought of a reasonable maximum number of attachments on an email I'd probably say 64."

We should not be talking of a "reasonable" maximum at all.

Rather, the maximum should be orders of magnitude away from the "reasonable".

At this distance, any arbitrariness quickly fades into meaninglessness.

Detached from the "reasonable" usage, the maximum then becomes informed by processing cost and complexity analysis, which are more concrete.

This assumes (and likely correctly assumes, in most cases) that 'reasonable limit' and 'probable system limit' are orders of magnitude apart.

I think that may not always be true though. Especially for highly variable workloads.

How many requests per seconds does your site handle on average vs how many can it handle? What about spikes?

If your previous peak is at 10 rps and you can probably handle 20 rps, when should you start dropping connections to keep the site up?

Most systems are designed to have small margins, for cost purposes, so I imagine this sort of situation might come up quite a bit.

I would disagree, a sensible limit that cannot overwhelm the machine even if a multi threaded server is processing its maximum number of emails, should be enforced.

The correct response to a breach of this limit would be for a reply email to explain the limit and why the email was rejected.

The user then could send multiple emails with their attachments, and the system can be sized to handle e.g. 24 threads processing 64 attachments of a maximum size of e.g. 4096kb

That's how you ensure a system sized to your hardware and ensure maximum throughput for all users.

"That's how you ensure a system sized to your hardware and ensure maximum throughput for all users."

Sure, I think we are actually in agreement. That's exactly what I meant by "processing cost and complexity analysis".

If you choose an arbitrary limit orders of magnitude above normal use, then you probably don't have any protection. Most systems are scaled to reasonable use, so an additional 1000x load in a dimension could bowl over the system.

Even defining "normal use" is intractable. For instance, most docker layers are a few MB, but some people are deploying 3P software packaged as a container with 10 GB in a single layer. You can't fix their container. They can't fix their container. Your definition of reasonable changes, and you bump your maximum to 1 TB. Then someone is trying to deploy docker containers that run VMs, which have 1.5 TB images. It's to interface with legacy systems that are infeasibly difficult to improve. But the vhd is a single sile, so now you have a single layer maximum size of 1.5 TB. But since the 10 GB body size is a possible attack vector in and of itself, what's the security benefit of having any maximum size limit at this point?

It's the wrong approach. Instead, your system should gracefully handle objects of arbitrary size. Security should be enforced by cryptographically enforced access controls and quotas.

Never done before (also i pick magic numbers from the air) but probably the idea is to look at Statistics of the data distribution and pick by Pareto the upper bound that cover more?

Is this data available? One simple one: How much long must be a name in a database field?

Some sort of lazy or other on-demand evaluation can help a lot here. You set the limits large, and then only evaluate the part of it that's actively being accessed.

> Every loop should have a limit. There should be no unbounded object allocation.

I’ve recently created some mini-languages (DSL’s) for everyone users to control various software and I’ve found that using a synchronous transformational language[1], which has no unbounded loops (or anything) and is therefore guaranteed to complete, and which runs in a stop-the-world fashion (ie each invocation acts as if atomic, whether it actually is or if implementation tricks like transaction rollback + retries are used) makes it easier both to develop and for end users especially ones who aren’t programmers. I’m a huge fan of this approach for correct and secure software.

[1] https://en.m.wikipedia.org/wiki/Synchronous_programming_lang...

I put limits on things in the D compiler. One thing is certain, I'll get a bug report because the limit was exceeded. Who'd have thought programs would be built with a hundred thousand symbols in them? Or that 65,000 lines in a source file isn't enough?

>Or that 65,000 lines in a source file isn't enough?

I would hope that was from auto generated code somehow..?!!

I've long since stopped caring why people do that, I just fix it so their programs work :-)

One of the things I learned working on software with millions of users is that ANYTHING that does not have a limit will eventually be abused by someone - often not even maliciously.

Got a feature that lets users add multiple additional options? Someone will use it to add 10,000 options, and now pages which display them in a <select> widget will slow to a crawl.

"Anything that does not have a limit will eventually be abused by someone."

There should be a law named for this.

What about Limit's Law?

The Law of Limits would probably be better. X's Law only really works well as a name if X is itself a name. Law of X is better when X is a topic.

> Every loop should have a limit. There should be no unbounded object allocation.

See also total functional programming, which applies the same logic to runtime.

Reply on news.ycombinator.com