Besides, you do still need to deal with the situation where the character shows up in your data; using an uncommon codepoint just makes it much easier to ignore until it blows up in your face.
JSON doesn't support binary data without conversion to string first.
It is for this, but if we're already declaring a restriction to JSON then we're likewise declaring a restriction to unicode. All delimited data requires escaping if your delimiter can be part of a no-delimit sequence. At least a character explicitly intended for delimiting isn't part of any no-delimit sequences representable in unicode whereas \n is.
U+1E is a Unicode code point, and therefore requires escaping inside of a ‘no-delimit’ sequence of Unicode characters.