Comment by lucideer | Hacker News Reader

Comment by lucideer | Hacker News Reader

Parent Full thread

There are 3 correct ways (and 1 incorrect way) to secure your strings in a web application:

1. Output escaping (server-side)

2. Output sanitization (server-side)

3. Output sanitization (client-side) - i.e. this standard

4. [Wrong] Input sanitization (server-side) (not to be confused with input validation, which is good but unrelated to this conversation)

Without boring with the details of why 4 is wrong, normally you should choose either 1, 2 or 3 depending on different applications:

(1) should generally be used in almost ALL cases. Contextual escaping is orders of magnitude more secure than sanitization, and is usable by any application not rendering user-provided rich text.

Furthermore, most/many user-provided rich-text applications use a DSL like Markdown which sidesteps the need for sanitization, so you can use escaping for these cases as well.

(2) should only be used if you are rendering user-provided rich-text. These cases are quite rare, and chances are if you need this feature you have a large application with lots of experienced engineers and you have the resources to put in place appropriate caching mechanisms.

(3) is the same as (2) but offloads the work to the client (no caching necessary) and has a resultant negative impact on the performance of your app (client-side UI latency). Making browsers fast is already a gargantuan challenge for browser-makers; adding client-sized sanitization to the mix will exacerbate this.

But mainly, this new standard will encourage inexperienced engineers to use (3) where (1) is much more secure for most applications.

5. Use proper API instead of escaping. (client-side) Say you assign your non-static data to el.textContent instead of concatenating strings into something that's to be interpreted as markup/code. Or you use DOMParser to parse markup safely, and then extract data from it or prune it using a whitelist via regular DOM API.

5a. Block all access to unsafe ways to use strings as code/markup via trusted-types CSP. (at least during development)

Conceptually the same thing you do with SQL server side. Parametrize the queries, instead of escaping and pasting into the query.

Great point!

It wanted to edit the comment to change (1) to (server/client) but I passed my edit timeout.

I would include your (5) within (1). `textContent` and other DOM methods like `setAttribute` are effectively secure output-escaping on the client.

Your (5a) is an excellent extra measure. In this area, I'd also add security-focused linting for (1) and (5)–e.g. for (5), to ensure secure DOM methods are used, I use Mozilla's `eslint-plugin-no-unsanitized`[0] plugin for all my personal & work projects.

[0] https://github.com/mozilla/eslint-plugin-no-unsanitized/

I think that's a bit missing the point of why sanitization is wrong (e.g. 2 & 3). The core problem is that blacklisting elements is really error prone. People frequently find ways around it.

You really shouldn't do "block this element, allow this element" type sanitizing ever.

What you should do is:

1. Parse the input into an AST that only allows safe nodes. I.e. the AST should have no `<script>` node at all. 2. Write it back out as HTML.

It's more work.

> You really shouldn't do "block this element, allow this element" type sanitizing ever.

> What you should do is:

> 1. Parse the input into an AST that only allows safe nodes. I.e. the AST should have no <script> node at all. 2. Write it back out as HTML.

Firstly I don't see the difference between what you say you shouldn't do and what you say you should do. This is what a sanitizer does: it parses the input into an AST that only allows safe things. It does "block this element, allow this element" on an AST.

So I don't understand what things you're contrasting?

Secondly, I guess this was just brevity, but disallowing script nodes is the tip of the iceberg when it comes to sanitization. You need to deal with a plethora of vectors involving scriptable & interactivity attributes, and other "active" non-script elements.

> Furthermore, most/many user-provided rich-text applications use a DSL like Markdown which sidesteps the need for sanitization, so you can use escaping for these cases as well.

Markdown does not side-step the need for sanitization. You can embed HTML inside markdown, so you actually need sanitization for exactly the use case of rendering markdown on the client.

> Furthermore, most/many user-provided rich-text applications use a DSL like Markdown which sidesteps the need for sanitization, so you can use escaping for these cases as well.

Be very careful about that assumption for Markdown. Many markdown processors will pass html tags through I modified so you could have a script in Markdown.

Good point. Maybe I should've said BB :D

I'll bite - why is 2 right but 4 wrong? Is it because it's not reversible? Is it in case your sanitization rules change?

Firstly, to distinguish sanitization -vs- validation: sanitization retains information lossily, validation throws. Validation may increase security, but the primary motivator is stability (avoiding unexpected state) and UX (relevant error messaging).

Input sanitization is intended purely as a security measure but is at best insufficient and usually reduces system security.

The first problem is that input sanitization is sanitizing without context: you don't know what threats you're fighting because threat vectors of this variety target the output format. Will your input variable be included into a HTML template, an Atom feed, saved to some file that's parsed later, sent in a JSON API, templated into a style or script block, used in an SQL query. The possibilities aren't known at input (or if they are they will change as your application grows) so protecting against all possibilities isn't viable. Often people html-escape or blindly html-strip their inputs, regardless of whether they'll ever be used in a html template.

Given the above you might think input sanitization is at worst useless, inefficient, but not harmful to security, but that brings us to the follow-up problems: lossiness & unknown state.

If you're doing input sanitization, you're not doing input validation (at least not properly). Input sanitization is about accepting lossy values (threats removed) when you receive unexpected input. That means you're accepting unexpected input, which leads to all the problems input validation is designed to protect against.

Finally, there is of course double-escaping. As mentioned above, people often html-escape as part of input sanitization. This essentially disables your ability to do reliable secure output escaping because that leads to double escaped values in output (or horrible double-escape-reversal hacks in output templating code). Generally you want to be securely output/escaping everything, which means you want a system where you can rely on always receiving clean unescaped values into your output template.

I have encountered too many systems using a good, secure html templating library with output escaping baked in by default, where devs had to disable output escaping explicitly because some inputs were already pre-escaped. In theory you can keep track of which vars need raw output and which vars don't, but that's an unmanageable mess in practice. And its impossible to automate enforcement.

Or in other terms: Store Raw, output safe means you can improve output safety over time. Storing safe locks you in to that mutation.

That's a really nice succinct way of putting it. Will steal.

Just don't steal my weird capitalisation hah

Reply on news.ycombinator.com