Hacker News Reader

Comment by lukaseder | Hacker News Reader

You might enjoy PostgreSQL, then. It supports a variety of "weakly" structured data types, such as JSON, hashmaps (key/value types, also called hstore in PostgreSQL).

Many databases also support XML data types. And of course, you can always store unstructured data in CLOBs and BLOBs

View on HN

Yes, but usually you don't want to use these 'weakly structured' BLOBS if you have an alternative. Relational dbs have some weak points but structured data storage is not one of them. I can't imagine a database without a fixed schema, it always exists even if you don't have to declare it upfront.

Actually, you read my mind! I'm currently working on a Rails project that uses PostgreSQL with hstore for semi-structured data such as user information, configuration, etc. Haven't gotten too far into it yet, but it looks promising. :)

What I was trying to imagine in my post, though, was some hypothetical DB where all this would be abstracted away--i.e., it would expose a NoSQL-like interface for all data, while working behind the scenes to provide SQL-like speed for frequent queries. Hence the analogy that it would be the "tracing JIT" of databases.

I've been thinking along similar lines, I think. One problem I see with data in RDBMSs is that data gets transformed in the process of importing it to match the schema, and this transformation loses information, i.e. it's hard to recover the original meaning and context, because it's been contorted to fit the schema. Schemas tend to need to evolve, and so you'd like not to be stuck with the assumptions of schema 1.0 forever. Data isn't that big these days in relation to the speed of computers, so you could keep the data in its original form and leave open the ability to "reimport" to fit the current schema as needed. If you're building a website, for instance, then your web app is going to be doing a limited number of queries over and over. If you start from the queries, you can infer things about how the data should be organized, e.g. which indexes are needed. In fact, if you took a query and pre-generated all the possible results of that query and saved them, that's basically an index, the one that fits that query. The inputs to this hypothetical database would be the original unprocessed data, the set of queries you need, and mappings to transform the unprocessed data into queryable form, i.e. the equivalent of a schema. The inputs apart from the data proper would also be considered as data.

Reply on news.ycombinator.com