Portable Text – JSON rich text specification for content editing platforms
github.com
github.com
If you want to build a JSON for text, then start by considering how to map XML to JSON. Many attempts at that exist, most terrible because the author didn't understand ordering requirements of child nodes (in XML they are ordered as they appear, but in JSON that's only true of arrays). The only reasonable way to map XML to JSON is to use arrays for all child nodes of any node, using objects at most just for things like node attributes and PIs.
I've done a fair bit of XSLT and jq programming. I understand the appeal of JSON here: jq is much pithier than XSLT/XPath! But first you have to get a bunch of things right that XML already got right.
EDIT: Once you have a very complex document schema, its mapping onto JSON will complicate any jq programs to the point where the advantage JSON has over XML as to DSLs disappears. XML simply is the right tool for documents.
We programmers love simplicity. The selling point of JSON was that it was really simple compared to the pile of specs which XML had become. But after a while it turns out people actually needed the features the XML standards provided: validation, querying etc. And a similar set of specs grow around JSON.
My guess for the next revolution: Some people figure out it is really annoying to have to quote keys in JSON and that JSON does not express expressions or DSL's very well. So s-expressions comes into fashion. And we get sexpr-schema, sexpr-query, and at some point someone will propose sexprs as a standard for structured hypertext.
Why do you think so? Is it because most structured texts are Trees?
If so I understand the appeal. Because hey, HTML is perfect isn't it? Except, it's really not. While HTML is great for rendering, it sucks at WYSIWYG editing.
Example:
<p> This is <span style="font-weight:underline;">a <b><i>formatted </b>text</i>content </span> </p>
The funda is - rendering is easy with trees, nodes and children. Editing though is easier with working with indices than traversing trees.
https://prosemirror.net/docs/guide/#doc.structure
http://marijnhaverbeke.nl/blog/prosemirror.html#the-document
These write-ups are interesting reads on the topic.
JSON and XML are both difficult to edit by hand, so don't do that. JSON, arguably, is harder to edit by hand because there's no CDATA, so you have to know how to escape characters, especially newline.
Both are tree-like structures. Why would one be better than the other? Well, XML has a long history of use and development for text documents, whereas JSON does not, so right there, that argues against JSON.
Does XML suck for WYSIWYG editing? I don't see why it would suck more than, say, TeX. LyX -a WYSIWYM editor for LaTeX- exists, and it can output XHTML using a LyX-specific schema. I'm sure LyX could easily have been built using an XML format from the get-go. You've not made any arguments as to why XML suck for WYSIWYG editing -- the example you pasted surely is meaningless in a WYSIWYG editing since that kinda implies having... a WYSIWYG (GUI) editor. And surely JSON will take more ceremony to express 'bold and italic' than XML, and not much less ceremony than XML to express 'use this specific font' (where the font's name alone is lengthy enough).
All you've done is expressed a preference for encoding details that no one really cares about, and you may not have stopped to really learn and understand XML, so your preference is leading you into making a number of mistakes the most important of which is NIH reinvention. Each NIH reinvention adds a fair amount of cognitive load for the rest of us, and adds development and maintenance burdens that never go away. Please don't do this. Innovation is very important, of course, but don't blindly reinvent to innovate.
I'd still stand by my opinion on not-making formatting information as separate tags and blowing the tree one level deeper. Preserving formatting options as index based properties would make editing APIs simpler and convenient.
Again, this is nothing in regard to the original comment. Just putting it out there.
In content editable, the visual information is not one to one transferable to a single html document, which, in turn, means not transferable to a single XML representation (and in all likelihood, the html is what is used as the representation, not a transformed XML document).
With Json representation , the content editable method is no longer the path of least resistance. Building out a proper renderer (whether in canvas or dom) means you end up removing most of the content editable issues.
https://medium.engineering/why-contenteditable-is-terrible-1...
Nobody says you have to operate on it in that format. You read it into internal structures and output those internal structures to HTML. I don't expect an editor to maintain a 1-to-1 relationship with the storage format while operating in memory.
In fact, even with HTML you could remove all the nesting from formatting blocks on load and either put them back on save or leave it flat (assuming you use CSS and not semantic style tags).
> Why do you think so? Is it because most structured texts are Trees?
XML allows for mixed contents. If I am reading this proposal right to change
<span>Some <b>formatted</b> content.</span>
<span>Some <b>forma<i>t<i/>ted</b> content.</span>
as the "text" field cannot contain childrens.
wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww ->
{ "type": "doc", "content": [ { "type": "paragraph", "content": [ { "type": "text", "text": "wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww" }, { "type": "hard_break" } ] } ] }
and with bold in the middle
{ "type": "doc", "content": [ { "type": "paragraph", "content": [ { "type": "text", "text": "wwwwwwwwwwwwww" }, { "type": "text", "marks": [ { "type": "strong" } ], "text": "w" }, { "type": "text", "text": "wwwwwwwwwwwwwwwww" }, { "type": "hard_break" } ] } ] }
This is not to say that is is a bad format, essentially XML does the same from an API level and maybe being explicit about it is good, but for sure is feels a bit different (again, do not want to express a better or worse).
Overall I am happy from the simple fact that it does not have the overused "Markup Language" in the name.
If I add your example into my XQuery processor, I get: "Different start and end tag: <i>...</b>."
Also, HTML != XML.
An HTML parser may forgive that error, an XML parser clearly does not.
Also, with JsonML there is a lossless mapping from XML to (array based) JSON (and vice versa), which makes the project linked here, totally redundant.
#1) All major rich text libraries (Prosemirror, Quill, Draft, Slate, Trix & more) out their now have their own capabilities and their own JSON representation. But exporting content across these editors is a pain. You either have to use HTML or Markdown, both of which have formatting loss.
#2) This means when you develop an app around document creation you are essentially locked down with either the libraries' data model JSON, or end up writing migration code and parsers if you want to switch.
#3) What if word processors have an option to export their content into such a JSON representation, such that parsing or rendering these content in browsers wouldn't involve us working with HTML
Challenges:
What if view information like line-endings and pagination are part of the model? Coming up with a JSON standard for such cases would be very difficult / impractical?
Now, a serious question: does this have an advantage over the existing standards, apart from "is JSON-based"? It looks like the company that made this decided to open source it, which is neat, but not very news-worthy.
There's already good tooling around XML-like dialects and the route from representation to rendering is much simpler.
There's a million reasons why XML is a bad fit, but one stands out: JSON is a first-class citizen on the web, and XML isn't. Once you parse this with the browser's built-in JSON parser, you're good to go; you have a JS object you can plug into an editor, or manipulate and display in all sorts of ways. If this were a custom HTML subset, or XML, the path would be much longer. JSON has many issues, of course, but ease of use isn't one of them.
Browsers parse XML too, y'know? It was, at one point, even more "first class" than JSON—why do you think browsers had an "XMLHTTPRequest" API, instead of just an "HTTPRequest" API?
> You're halfway into XML land at this point.
You mean... XHTML?
The ability to combine HTML tags (retaining their semantic meaning), with tags from your own, other XML namespaces (which you can define the semantic meaning of) was 99% of the point of XHTML.
People thought XHTML was supposed to "replace HTML" or "be the next version of HTML", and so hated it (because they'd have to fix all their existing HTML-authoring tools to make them generate valid XML.)
But no, XHTML was never supposed to entirely supplant HTML; XHTML was just supposed to be a syntax for including HTML-markup-ed text as part of the content of a larger XML document.
And XHTML is still a valid thing to use today. WHATWG is even developing XHTML5 to mirror HTML5.
Really, XHTML should not be considered "some past version of HTML, a bad diversionary experiment like New Coke that we've since gotten back on track from", like most people see it. Rather, XHTML was and is a separate tool, useful in particular domains for acting as a drop-in sub-schema to larger XML document formats that need to represent text markup within them. XHTML provides not only the DTD, but all the semantics for how those elements should translate to rendered results—while still allowing you to do whatever you want, XML-wise, inside and outside and around those elements.
XML is a far better format for this; it is designed specifically for mixing text content with markup. This format is incredibly hard to read and debug. Once you've parsed this JSON, you're still left with a complex stringy structure that you then have to parse again anyway. The value of being in JSON is incredibly low.
I'm not sure why you think this particular JSON schema is a "complex stringy structure". XML is a complex stringy structure! There's conceptually and semantically zero difference between <node style="normal">text</node> and {style: "normal", text: "text"}, other than the fact that the former requires an API to read and manipulate, and the latter doesn't, from the point of view of JavaScript code. XML has a specific formalized data model of elements, attributes and so on, but this is just a graph representation: You can lossly encode an XML DOM in JSON, and vice versa.
The argument in favour of XML should revolve around the ecosystem of tools (XPath, XML Schema, namespaces, etc.) that allow you to work with it, but so far the comments here haven't conclusively highlighted these features as benefits. They also seem to ignore that with the exception of legacy enterprise software, the web world has largely (not entirely, but largely) moved away from XML. There's a reason we're using JSON over REST instead of SOAP. XML isn't the lingua franca of the web that W3C seemed to assume it would be during the early 2000s; JavaScript/JSON is.
But this format will require parsers to take the resulting objects, arrays, and numbers and reconstruct something very close to XML's data model anyway.
> XML's data model is complex.
So is this document format. With it's special "_type" keys and specially named nodes. XML is simpler since attributes and children are a native part of the format and the API.
> There's huge value in your document model also being your in-memory model.
Are you suggesting this exact model be the in-memory model? Because that would be pretty non-optimal. You'd likely want your in-memory model to be more powerful than a tree of dictionaries, arrays, and strings. Therefore parsing and storing an XML document on load/save is not a big deal. In fact, given the existence of an API that already handles attributes and nested nodes, it might even be easier.
> XML has a specific formalized data model of elements, attributes and so on, but this is just a graph representation: You can lossly encode an XML DOM in JSON, and vice versa.
Exactly. So why use a format that it obviously terrible for mixed text and markup? For any reasonable sized document in JSON will be impossible to follow yet XML would be comparatively straight forward.
> There's a reason we're using JSON over REST instead of SOAP. XML isn't the lingua franca of the web that W3C seemed to assume it would be during the early 2000s; JavaScript/JSON is.
Basically SOAP is an over-engineered solution to web RDP. It started out as XML-RPC which was far simpler and far more constrained like JSON. You could easily encode SOAP in JSON if you wanted. You're blaming the underlying technology for when it's used inappropriately but this is an example of JSON being used inappropriately! It's an over-engineered solution to a problem solved simpler by a different technology. "XML bad / JSON good" doesn't tell the whole story.
I'm pretty confident that this is actually the intended goal, particularly for web-based editors.
Most "modern" web-based rich text editors work with an internal schema similar to this one, even when most of them support html/xml import/export too. In today's web ecosystem, it is hard to do much better than "javascript structure that gets diff-rendered to the DOM by a patching algorithm on updates".
I think this is just a case of someone saying "hey, wouldn't it be great if all these editors used the same schema instead of very similar ones with minor differences?"
The data model, yes, though not the visual model. Sanity [1], which uses this specification, uses it for its rich text content.
Sanity is a document store with a collaborative, real-time editing UI similar to Airtable, but self-hosted and open source. When you're editing some text in the rich-text editor (which uses ProseMirror internally, last I checked), the editor is working directly on this data model, and produces patches against this data model that get synced with other clients. Similar to OT, except it uses a simpler git-like approach to rebase patches and converge the state.
There's another, less obvious reason it's JSON. In Sanity's document store, every document is a structured object that is read and written as JSON. Inside each document, Portable Text fields are also stored as JSON, not as a string or a binary blob. This content also is indexed and queryable. For example, let's say you're creating a wiki app; each wiki page is a document, with a title, body, and so on. The body is a Portable Text field. With this, you can find all wiki pages that link to another wiki page by running a query such as *[references("richard-feynman")]. You can do things like extract images, get the first paragraph, etc., all using the query language.
All technically possible with XML, of course, though I personally wouldn't want to go there.
(Disclosure: I work on Sanity's content store tech, but I don't work on Sanity itself.)
That's fine. APIs are good. The problem with XML is that the APIs never got any love from the end-user perspective. They are horrible to use, (seemingly) overcomplicated and the ones I've used, inconsistent.
I've seen one or two nice Python wrappers (forget names, been a long time since I worked with XML) that smoothed out the Developer Experience (DX), but that's it.
For me, this is the reason as a programmer I prefer to work with JSON, even though as a data author and maintainer I prefer XML at every level.
And JSON doesn't?! Sure it does. E.g., I regularly use the jv API in libjq. I wouldn't want to use JSON in any other way than via an API, honestly.
You could also argue that XSLT/XPath are part of the XML API / ecosystem, much the way I think of JSONPath / jq as part of the JSON ecosystem.
And yes, the world has moved away from XML... for non-document purposes. XML was a poor fit for encoding non-document data. SOAP's use of XML to encoded non-document data sure sucked, though it died of other things too (e.g., REST becoming more popular and being more appropriate in some ways). That's why we have things like protocol buffers, flat buffers, etc. Some even remember XDR, NDR, ASN.1 and its many encoding rules. Some even remember those and curious enough to have noticed that protocol buffers is DER reinvented.
JSON is much more appropriate for non-document data than XML, and unlike XDR/NDR/<various>buffers/DER/BER/CER/XER/PER/OER you don't have to agree on a schema a priori, which makes JSON very useful in REST APIs. But you could still define REST services in relation to schemas and then use OER or flat buffers to get very efficient encodings.
At the end of the day it's all the same data no matter how you encode it, provided the choice of encoding doesn't force you to give up some metadata or force you to express it in a convoluted way.
The point isn't that you can't do with JSON what you can with XML, or that you can't do with XML what you can with <various>buffers. The point is that XML has decades of evolution baked in already. You should not reinvent a document format without being very explicit about what's wrong with XML, what's missing that can't be added, and what you're willing to throw out, otherwise you'll find yourself replicating that evolution.
const foo = {
bar: n,
children = users.map(({name, id}) => ({name, id});
}
foo.x = 42;
const foo = document.createElement("root");
foo.setAttribute("bar", `${n}`);
for (const ({name, id)} of children) {
const child = document.createElement("child");
child.setAttribute("name", name);
child.setAttribute("id", id);
foo.appendChild(child);
}
foo.setAttribute("x", `${42}`);
JSON and XML are both encodings of two different formal data models. The difference is that JSON is native to JS. XML requires an API to build a safe and valid document. There are simplifying wrappers around the DOM, but they'd be yet another API on top of an API.
Also note how XML is string-based. In order to get any type safety at all you have to use XML Schema, which only works with a data consumer that also knows XML Schema. So given <foo x="1"/>, there's no way to tell arbitary consumers that x is, in fact, a number.
There are undoubtedly toolchains that do higher-level manipulation of XML data so that you can serialize and deserialize rich, type-safe data, but the fact remains that you need such a toolchain, in every single language that is to deal with this data.
More than anything, XML is trivially extensible, but anything you do with JSON for documents will require that you reinvent just about everything that XML already has (e.g., namespaces, schemas), and as with all reinventions, you'll probably do it badly.
I've said elsewhere here that jq is so much pithier than XSLT, and that's true. But once you have a very complex schema to represent in JSON what would have been a simpler schema in XML, jq will no longer yield short, simple, pithy programs. The advantage of simplicity that JSON seems to have here over XML is imaginary and ephemeral -- I wouldn't count on it.
JSON is not annotated text, it is structured data that contains strings. It is a minor difference if you only use tools to manipulate it though.
Are there similarities between Portable Text and how rich text is represented in e.g. ProseMirror and SlateJS and DraftJS and other rich text editors?
Is there a way that I can build my own custom XML tags and prototype my own custom XML document format? Something like:
<CustomGraphXMLtag>somevaluehere<CustomGraphXMLTag> <somenewtag2>some other value</> ........
Can I do this with Apache OpenOffice SDK or any other way?
EDIT: But still, no please. This isn't even remotely close to on the same level as XML is, and we don't need XML to be reinvented.
Also another thing that has bugged me about "Rich text" since the Word 95 days was that something seems to be rotten in the data model in most implementations.
The main symptom is that applying styles to text just doesn't behave rationally. I want to select a certain region but I wind up selecting something slightly different. The mouse pointer moves in the wrong direction, etc.
Related to this it seems most WYSIWYG editors for HTML are buggy for the same reason. Dreamweaver used to work 10 years ago but now it can't get out of it's own way on my overpowered laptop and Adobe Support is clueless about it.
Will this help?
{
"_key": "a-key",
"_type": "markType"
}
{
"a-key": "markType"
}
{
"style": "h1",
"children": []
}
"h1": {
"children": []
}"style" is optional, so it doesn't make sense for "h1" to be a key.
More importantly, JSON is harder to work with when a key can be anything. A schema is more uniform and easy to make strict if all keys are reserved. This also makes it easier to map to data structures used in languages that have struct-type types. Think of the JSON objects here as structs, not key/value pairs:
type Block {
Style string
Children []Block
Type string
}
What if we had a limited kind of markup as part of Unicode, such that all Unicode text could be "marked up", entirely-portably, such that as long as you aren't rewriting the Unicode representation, you're preserving the markup?
Unicode already handles many kinds of markup. We just don't tend to think of these features of Unicode as "markup"—even though many of them have one-to-one equivalents in markup languages!
Most obviously, you've got newlines, the plaintext equivalent of the HTML <br> tag. If Unicode was "just" about representing text, newlines (and horizontal/vertical tabs, non-breaking spaces, zero-width non-joiners, etc.) wouldn't be part of it. These aren't text; they're instructions to a text layout engine, for where to put the text. They're markup, just as much as an HTML <center> tag is!
Less obviously—but more flagrantly—Unicode has the BIDI LTR/RTL instruction codepoints, and support for storing and representing a https://en.wikipedia.org/wiki/Ruby_character gloss of a given character-sequence embedded alongside said text.
And because of this support, Unicode font rendering engines already have everything they need to be able to render a given "meta-instruction" codepoint that makes a given "run" of characters bold, or italic, or small-caps, or strikethrough; or to indicate that a given run of codepoints should be considered a list item, or even a semantic "paragraph" or "heading."
It's my understanding that the maintainers of the Unicode standard would balk at standardizing this kind of markup within Unicode. But, if all this is possible at the Unicode-parsing level, then why not just create a separate working group who will maintain such markup codepoints as a Unicode Private-Use Area, and then get font-rendering-engine vendors to support said markup codepoints?
---
Side-note: the result would be a lot like creating Unicode-codepoint equivalents for ANSI escape sequences, wouldn't it? In fact, I would imagine that pretty much every common ANSI "Select Graphic Rendition" escape sequence would make sense as a Unicode markup codepoint, save for maybe being translated to prefix-length or per-grapheme-cluster-paired forms rather than being modal switches.
And oddly, I think that such SGR-equivalent codepoints would be within the spirit of Unicode. Unicode wants to allow for the digitization and standards-based preservation of all historical text. Well, how are you supposed to preserve the text-art sent by an 80s BBS, without being able to represent (at least) fg/bg color changes? Unicode already decided to standardize the DOS box-drawing characters for exactly this same reason!
It would be pretty neat though.
Unicode tag codepoints simply cannot be reliable.
Nor can Unicode tag codepoints provide sufficient functionality to replace XML (or HTML, or ...). Even the subset they can replace/displace would require revamping XML/XSLT processors in order to make XSLT able to see them -- that's a terrible imposition on the rest of the world.
Some of us hate UTF-16, but there it is and we can't get rid of it.
Others hate UTF-8, and.. the same thing applies.
XML exists, it's widely used, it's not going away no matter how much anyone dislikes it. Moving parts of HTML or XML into Unicode will not work for the reasons given above. Please dedicate your energy and ideas to innovating in other ways than this.
You're assuming a representation that encodes such markup as ["start of bold" codepoint], ["end of bold" codepoint]. I agree that this would be bad, for the same reason that the Unicode "language" tags were bad.
There are other representation options, though.
You can instead do exactly what Unicode's Ruby-character support does, and just make a markup annotation last for exactly one grapheme-cluster. A run of bold characters, then, is a run of grapheme-clusters where each grapheme-cluster has been individually bold-annotated. There's no real way to "break" that. It's inefficient, of course, but the point of it isn't to be a universally-optimal format, but a markup-format-of-last-resort when there's no higher-level markup layer in the document format to hold styling information.
To be clear: there's demand for a "markup-format-of-last-resort." In fact, people are already abusing other Unicode features to create one, and are quite happy with the resulting "styled" Unicode documents, which are—horrifyingly—proliferating across the internet and breaking semantic machine-readability for any future archival uses of that text.
If you can't guess, I'm talking about people replacing the Unicode codepoints of their text with entirely-different Unicode codepoints, that just happen to already look like the "styled text" version of the text they want. For example, you've got 𝓽𝓱𝓲𝓼 𝓽𝓮𝔁𝓽, ʇɥᴉs ʇǝxʇ, t̶h̶i̶s̶ ̶t̶e̶x̶t̶, etc. These are the outputs of "text generators" (online IMEs, essentially) that take your Unicode plaintext input text, and output Unicode plaintext output text that is "styled."
Now picture a Unicode feature to obviate such "text generators"—where, instead of two codepoints ["𝓽"]["𝓱"] that are both 1. impossible to type on a keyboard and 2. not actually the same characters semantically as ["t"]["h"], you've instead got four codepoints: [next-with-italics]["t"][next-with-italics]["h"]. You typed ["t"]["h"], and then selected them both and hit the "I" button in your editor, and then copied the result, and that's what's now in the copy buffer.
Nothing would break—text editors already understand how to keep Unicode combining diacritics together with their base codepoint just fine, and this is just another use of the same logic.
Similarly, the NFD representation of [next-with-italics]["t"][next-with-italics]["h"] would just be... ["t"]["h"]. Stick your four-codepoint sequence in a DB fulltext field, and it'll index it no problem.
> Nor can Unicode tag codepoints provide sufficient functionality to replace XML
Oh, I would never suggest that. Structural markup has no place in Unicode. I just think that there is a thing that is a subset of "rich text" but a superset of "Unicode plaintext" and maps to exactly the sort of text-styles that a publishing editor laying out InDesign flows, would let their contributing writers use in the RTF documents data-bound to those flows; or which a blogging engine like Twitter would let their users use to style their tweets.
Like I said, this doesn't belong in the Unicode standard. Such a format can be considered an additional format layer between Unicode plaintext and a document format. It just so happens, though, that you can take advantage of the Unicode-rendering capabilities of most rich-text layout engines, to support such markup. Same as how the VT100 spec took advantage of the logic that already existed to respond to newlines and such, and introduced inline escape sequences—nominally "a higher layer", but in practice a thing that is conflated with the layer below it, such that any terminal program that claims to output "plaintext" might actually be outputting plaintext-with-ANSI-escapes without mentioning it.
BTW, games with BS (backspace) and overstrike for bold and underscore for underline still work in Unix terminals... That might be the best argument for having some stylistic composing codepoints in Unicode... after all, ASCII was designed in part to be usable with BS for expressing diacritical marks (accents, umlauts, cedilles, ...), making it a variable length internationalized encoding! But still, then less we complicate Unicode, the better.
If you can use XML, by all means, please do use it. If you're designing a document format, and there should be markup in your document, just use XML for your document format and then XHTML for the markup parts. It's what ePub does. (ePub is one of my favourite formats: it's just a slapping-together of a bunch of well-known technologies. It's the "buy rather than build" of document formats.)
I do think, though—separately from what your options are as a document-format designer—that there's a point in having a Unicode-level markup standard, and getting the font-rendering engines on board with it. And that point is democratization of styling control: forcing styling into places where the format specifier had no intention, way back whenever they specified the format, to add styling; and where the software stack has maybe ossified since, such that nobody's willing to introduce a new version that specifies a new format with styling (or where such an effort was tried but has long fizzled out.)
Want to add styling to email? No, not HTML email, pine(1) ain't ever gonna render that for you. I mean, all email. Email in an app in the GUI; email on the CLI in your terminal emulator. Email even in pine(1), without modifying pine(1), merely updating GTK and Cairo and then restarting gnome-terminal. Unicode markup would work entirely opaquely to the email client's intent to "have styling" or not. At the layer the email client is operating at, it's Just Text™, just like emoji are now Just Text™.
As far as I can tell, encoding the styling at the Unicode-codepoint layer, and then parsing it out at the {OS graphics toolkit, terminal emulator} text-rendering layer, is the only way to make that work.
Or, for another example. How about we insert some bold text into a TEXT-typed column of a Postgres DB row? Yes, the text is supposed to be bold. No, we can't change the type of the field to XML; that would break everything that currently queries that field. I want to just put bold text in one row, one time. Yes, sure, add an INSERT trigger to ensure that nobody can insert text with any other styles. (But if you're so worried about that, why don't you have that trigger already to ensure that nobody's inserting a newline, huh?)
A cute trick: make temp-file filenames not "foo~" but rather "foo"-with-a-strikeout-through-it. Yes, that'd be the file's actual on-disk name. You're saying you want your filesystem to use XML for the filename field?
(I mean, these are ridiculous, yes. The only actual use-case is obviously allowing people to use bold on HN.)
The situations, AFAICT, are often to defeat text filters.
While this might represent a large percentage of the "pre-baked styling" Unicode text in the wild by volume of text, there aren't that many people who have created all this text (most of the corpus is spam repostings of the same messages with randomly-varied styling.)
Rather, I think the majority of the authors of "pre-baked styling" Unicode text are simply people posting to websites like Twitter or Facebook, or a Disqus comments section, where styling is either not allowed, or doesn't have all the options the author wants.
Or, to put that another way: the authors are mostly teenagers who care more about the aesthetics of their text than its comprehensibility.
And that's surely the market that the Unicode consortium cares the most about—why else would we have so many emoji? ;)
Most of the Unicode markup features that already exist—other than newlines—form a backing layer: a standard interchange format for text, when it's not in the internal DOM+ropes representation that most text editors use. It's not a working representation, it's just a representation that text is rendered out to.
(In fact, for a lot of these existing markup codepoints, when there's one sitting around "raw" in a text field—rather than having been parsed into a text-editor DOM node—you can't even position the cursor and then delete it. Such codepoints just sort of "stick" to the grapheme-clusters/edit-field boundaries on one or both sides of them, such that deleting the grapheme-cluster will maybe "garbage collect" the associated markup, or maybe leave it there forever. The editor doesn't expect that character to be in its text, breaking the layout. It expects to have parsed it out, and then to put it back in place when it saves the document. The markup codepoints exist to be inline instructions for rendering a run of "baked" text; they don't exist to be a convenient data structure for text-editors to do editing operations against.)
So, the idea I'm proposing here, is not that you'd just "be able to type a <b> codepoint"; but rather that:
1. plaintext editors like Notepad and TextEdit—and HTML's plain <textarea> controls—would now have buttons for simple text styling (because now they can!)
2. even OS plaintext edit controls would allow styling of their contents, through an OS-universal styling palette (like macOS's "Fonts panel") that would apply to the cursor/selection of the active window (like it does for macOS rich-text edit controls.) It would essentially be part of the IME, like macOS's minimized-form "Character Viewer" palette is, or like dead-key entry is on X11.
2. WYSIWYG editors for rich-text formats like RTF and Markdown would move to evolved, distinguished storage formats (e.g. ".rtfu" = "text/rtf+umark") where the choice-of-format has the meaning of canonicalizing all styling that is possible using Unicode markup codepoints, as Unicode markup codepoints.
3. The original formats these new formats evolved from ("text/rtf", "text/markdown", etc.) would now be defined specifically as formats that have no knowledge of the meaning of Unicode markup codepoints, such that if a file containing Unicode markup codepoints were to be "Open As"ed the old format, the codepoints would render as �s. (But if you just open the document without an "Open As", the editor would heuristically determine from the fact that the file contains markup codepoints, that the file is actually "text/rtf+umark", and would insist that you "Save As" the file with the new extension.)
4. Each OS's neutral "rich text" drag-object/paste-object format would switch from "text/rtf" to "text/rtf+umark". All text-editing controls on OS apps would be required to both produce this as on drag/copy (even if it's not their storage format), and accept+parse this on drop/paste (even if it's not their storage format.)
5. Each OS's neutral "plaintext" drag-object/paste-object format would become "text/plain+umark"—i.e. plaintext that preserves styling. Thus, copying from a rich-text edit control, to a plaintext edit control, would get you resulting text with editor-specific styling stripped, but retaining the Unicode markup codepoints.
6. If the OS supports arbitrary-Unicode filenames, it should probably accept Unicode filenames with markup codepoints in them, and should probably render that markup when displaying the filename. (Consider: at least in Linux, filenames can contain newlines, and the OS renders those.)