Why can't you stream a JSON array?
Edit: Here's a (hastily created and untested) node.js example, even:
class JSONArrayStream extends Transform {
constructor() {
super({readableObjectMode: false, writableObjectMode: true});
this.dataWritten = false;
}
_transform(data, encoding, callback) {
if (!this.dataWritten) {
this.dataWritten = true;
this.push('[\n');
this.push(JSON.stringify(data) + '\n');
} else {
this.push(',' + JSON.stringify(data) + '\n');
}
callback();
}
_flush(callback) {
this.push('\n]');
}
}Editing, because I can’t reply: I was specifically going to mention SSE/EventSource but expected an immediate “not everything is a browser” response.
Editing the 2nd: yep, that’s kinda what I meant by “tricks” - essentially splitting out chunks to pass to the json parser.
- how can a client send an error message if it runs into problems mid-stream? You can invent a system, but you’re walking into an ad-hoc protocol pretty fast; why not use something others wrote?
- what if the remote end wants to interrupt the stream sender to say “stop sending me this” for any reason? For example, an erroneous item in the stream, or a server closing down during a restart.
- grpc supports fully bidirectional streams, interleaving request and response in a chatty session; how do you do this?
Not that the original article mentioned these. I bristle though when I hear the engineer’s impulse to “why don’t you just”-away at something.
“binary” is not necessarily a benefit, particularly for developer tooling/debugging
Streaming is one area it may have a benefit but honestly the other issues outweigh that possible benefit (and it’s not like there aren’t other ways to stream data to a browser without resorting to polling)