This is indeed a fun and challenging type of API endpoint to implement.

I built a streaming statistics server that ingests data from edge nodes and aggregates the data into a memory database.

The server has an API interface and one of the endpoints will stream all the data in either a compact or pretty JSON format.

example of the datasets: millions of unique unordered keys with each key containing a list of time series data.

The memory overhead to stream Gigabytes of data is limited to a couple of MB (buffers for encoding and writing).

I ended up having to write my own JSON encoder to achieve the small memory overhead, JSON formatting, and write performance.