How to safely escape JSON inside HTML SCRIPT elements
sirre.al
sirre.al
Kaminsky described a very simple and nearly-universal technique to deal with escaping/injection issues. Encode the embedded data as base64 and decode it on the client side. This projects arbitrary data into a fixed, known domain (generally `[a-zA-Z0-9+/]*`) which you can ensure is free from control characters. (You may need to use a particular variant to achieve this, eg for URLs the last characters used are generally `-_` because both + and / are significant in that context.)
After decoding, you can pass it to JSON.parse().
And yeah use URL-safe base64 when you do use it. -_ with no padding.
The advantage of the base64 technique is that it provides fewer degrees of freedom, and so is more robust to unforseen vectors of attack. It's defensive programming. But it comes at a cost of memory/bandwidth.
(This is related to the 'prototype pollution' attack, although searching that phrase will mostly give you information about the more-dangerous variant where two objects are being merged together with some JS library. If __proto__ is just part of a literal, the behavior is not as dangerous, but still surprising.)
type
This attribute indicates the type of script represented. The value of this attribute will be one of the following:
[...]
Any other value
The embedded content is treated as a data block, and won't be processed by the browser. Developers must use a valid MIME type that is not a JavaScript MIME type to denote data blocks. All of the other attributes will be ignored, including the src attribute.
https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/...
It "conflicts" in the same way noscript[1] and script "conflict" no? They're basically related features, but can't really be made exclusive because the mere act of trying to do so wouldn't work: as the link indicates, executing code in a !module browser reserves the type (requires a specific set of types) so you can't use that as a way to opt in !module browsers.
[1] an other fun element with wonky parsing rules besides
<script nomodule="true" type="module"></script>
https://html.spec.whatwg.org/multipage/scripting.html#attr-s....
My expectation was that this condition would have been reflected in MDNs documentation where it breaks the conditions for 'charset' and 'defer' out.
Perhaps the spec isn't the right tool for every job? That's why, for me, at least.
Yes, and you can write
<noscript>
<script>
...
</script>
</noscript>- escape `<` as `\u003c`
<script id="my-json" type="application/json">{{ escaped_json }}</script>
JSON.parse(document.getElementById('my-json').textContent)
<script language="JavaScript"><!--
// script contents
-->
</script>I guess people just generally don’t add those?
Still, to help me out, could someone clarify why this was down-voted? I don’t want to mess up again if I did, but I don't understand what that was.
Most people will opt for text to be optional with a link - unless they're showing their own product (Show HN). Because there is an expectation that you will attempt to read an article, before conversing about it.
I think most of the time people dont add a comment to submissions, but if they do its more of the form: I found X interesting because of [insert non obvious reason why X is interesting] or some additional non-obvious context needed.
In any case, i don't think there is any reason to worry too much. There was no ill intent and at the end of the day its all just fake internet points.
CDATA: https://en.wikipedia.org/wiki/CDATA
<![CDATA[
]]>
<script>
<![CDATA[
x = {"<!--":""};
]]>
<![CDATA[
{{json.dumps(["<!--"])}}
]]>
</script> Everything until the tag closer </script> is inside
the script element.
In fact, script tags can contain any language (not
necessarily JavaScript) or even arbitrary data. In order to
support this behavior, script tags have special parsing
rules. For the most part, the browser accepts whatever is
inside the script tag until it finds the script close tag
</script>.
As to:
why a script tag inside a comment inside a script tag needs
to be closed ...
When JavaScript was first introduced, many browsers did not
support it. So they would render the content of the script
tag – the JavaScript code itself. The normal way to get
around that was to put the script into a comment ...
Or did they always have two levels of script tag escaping but that behavior only got preserved when inside an HTML comment?
No other JavaScript behavior is different inside an HTML comment, and I’m still missing the connection between the HTML comment and the embedded </script> not closing the tag besides that they were two things that older browsers might have done.
It’s helpful to recognize that the inner script tags are not actual script tags. Yes, once entering a script element, the browser switches parsers and wants to skip everything until a closing script tag appears. The STYLE element, TITLE, TEXTAREA, and a few others do this. Once they chop up the HTML like this they send the contents to the separate inner parser (in this case, the JS engine). SCRIPT is unique due to the legacy behavior^1.
HTML5 specifies these “inner” tags as transitions into escape modes. The entire goal is to allow JavaScript to contain the string “</script>” without it leaking to the outer parser. The early pattern of hiding inside an HTML comment is what determined the escaping mechanism rather than making some special syntax (which today does exist as noted in the post).
The opening script tag inside the comment is actually what triggers the escaping mode, and so it’s less an HTML tag and more some kind of pseudo JS syntax. The inner closing tag is therefore the escaped string value and simultaneously closes the escaped mode.
Consider the use of double quotes inside a string. We have to close the outer quote, but if the inner quote is escaped like `\”` then we don’t have to close it — it’s merely data and not syntax.
There is only one level of nesting, and eight opening tags would still be “closed” by the single closing tag.
^1: (edit) This is one reason HTML and XML (XHTML) are incompatible. The content of SCRIPT and STYLE elements are essentially just bytes. In XML they must be well-formed markup. XML parsers cannot parse HTML.
"My momentary convenience trumps the man-millenia of effort required to protect billions of people from script injection attacks."
Edit: and "effort", please. The spec has a simple and clear note:
> The easiest and safest way to avoid the rather strange restrictions described in this section is to always escape an ASCII case-insensitive match for "<!--" as "\x3C!--", "<script" as "\x3Cscript", and "</script" as "\x3C/script" when these sequences appear in literals in scripts (e.g. in strings, regular expressions, or comments), and to avoid writing code that uses such constructs in expressions. Doing so avoids the pitfalls that the restrictions in this section are prone to triggering.
Backwards compatibility is easily and completely worth this small amount of effort. It's a one-liner in most languages.
[1]: https://uploadcare.com/blog/vulnerability-in-html-design/
Bullshit - Navigator and IE didn't have HTTP/2. I'm guessing you didn't use dialup where your external CSS or JavaScript regularly failed to load. You didn't add extra dependencies because IE would only had two concurrent connections to load files.
It's easy to criticize past mistakes from your armchair: but I suggest you try and be a little more fair towards the people that made decisions especially when overall HTML has been a resounding success.
Have you done even a single thing in the markup community?
Engineers hate bad compromises, and the core of engineering is making good compromises. Creating anything makes you your own critic.
In this link we can see the expectation that the HTML comment surrounds a call to document.write() which inserts a new SCRIPT element. The tags are balanced.
https://stackoverflow.com/questions/236073/why-split-the-scr...
In this HTML 4.01 spec, it’s noted to use HTML comments to hide the script contents from render, which is where we start to get the notion of using these to hide markup from display.
https://www.w3.org/TR/html401/interact/scripts.html
Some drafts of the HTML standard attempted to escape differently and didn’t have the double escape state.
https://www.w3.org/TR/2016/WD-html52-20161206/semantics-scri...
My guess is that at some point the parsers looked for balanced tags, as evidenced in the note in the last link above, but then practical issues with improperly-generated scripts led to the idea that a single SCRIPT closing tag ends the escaping. Maybe people were attempting to concatenate script contents wrong and getting stacks of opening tags that were never closed. I don’t know, but I suppose it’s recorded somewhere.
Many things in today’s HTML arose because of widespread issues with how people generated the content. The same is true of XML and XHTML by the way. Early XML mailing lists were full of people parsing XML with naive PERL regular expressions and suggesting that when someone wants to “fix” broken markup, that they do it with string-based find-and-replace.
The main difference is that the HTML spec went in the direction of saying, _if we can agree how to handle these errors then in the face of some errors we can display some content_ and we can all do it in the same way. XML is worse in some regards: certain kinds of errors are still ambiguous and up to the parser to determine how to handle, whether they are non-recoverable or recoverable. For those non-recoverable, the presence of a single error destroys the entire document, like being refused a withdrawal at the bank because you didn’t cross a 7.
At least with HTML5, it’s agreed upon what to do when errors are present and all parsers can produce the same output document; XML parsers routinely handle malformed content and do so in different ways (though most at least provide or default to a strict mode). It’s better than the early web, but not that much better.
There are two situations in which it does.
① XML syntax, which is absolutely still a thing:
data:application/xhtml+xml,<html xmlns="http://www.w3.org/1999/xhtml"><script>console.log( 1 > 0 && 0 < 1 )</script></html>
data:text/html,<svg><script>console.log( 1 > 0 && 0 < 1 )</script></svg>That ship sailed several paragraphs ago, when <script> got special treatment by the HTML parser. Too bad we couldn't all agree to parse <![CDATA[...]]> consistently, or, you know, just &-escape the text like we do /everywhere else/ in HTML.
<script>console.log("<![CDATA[Hello, this string content in a CDATA section!]]>");</script>
<![CDATA[Hello, this string content in a CDATA section!]]>
I tend to use them anyway, as sort of a HTML/XHTML polyglot thing, because deep in my heart I still think HTML should be valid XML:
<script>/* <![CDATA[ */
// my script here, and you *still* need to be careful not
// to include close-script or close-cdata sequences
/* ]]> */</script>
Never was, never will be. Just write XHTML instead.