Hacker News Reader

Comment by josalhor | Hacker News Reader

How efficient is this? Is JSON stored as a blob and parsed and filtered on each query?

In case of PostgreSQL there is json and jsonb. For SQLite, hexdump of the database shows text representation and seems to be stored like json than jsonb. I am not aware of the full design and source code but it seems some functions parse and cache the JSON representation.

https://github.com/sqlite/sqlite/blob/a0318fd7b4fbedbce74f13...

https://www.postgresql.org/docs/current/datatype-json.html

> The json and jsonb data types accept almost identical sets of values as input. The major practical difference is one of efficiency. The json data type stores an exact copy of the input text, which processing functions must reparse on each execution; while jsonb data is stored in a decomposed binary format that makes it slightly slower to input due to added conversion overhead, but significantly faster to process, since no reparsing is needed. jsonb also supports indexing, which can be a significant advantage.

> In case of PostgreSQL there is json and jsonb. For SQLite, hexdump of the database shows text representation and seems to be stored like json than jsonb. I am not aware of the full design and source code but it seems some functions parse and cache the JSON representation.

Wouldn’t a jsonb-type storage require a new storage mode for sqlite, which would be a major architectural change?

JSONB derives (at least logically and historically) from hstore, so postgres had a history of structured type stores. Not so for sqlite.

jsonb and hstore are both just an encoding format for the data within a single field and do not affect either the architecture nor the storage model of the database.

https://www.sqlite.org/json1.html#interface_overview mentions that a binary encoding of JSON was experimented with, but no significant speedup or space savings were observed. If the functions are parsing and caching JSON that might explain why the performance is so close.

> SQLite does not (currently) support a binary encoding of JSON. Experiments have been unable to find a binary encoding that is significantly smaller or faster than a plain text encoding. (The present implementation parses JSON text at over 1 GB/s.) All JSON functions currently throw an error if any of their arguments are BLOBs because BLOBs are reserved for a future enhancement in which BLOBs will store the binary encoding for JSON.

Yes.

If you look at the sample data section, you’ll see an example of the data they’re using.

It’s mainly to parse and work with json data that’s stored.

Reply on news.ycombinator.com