Hacker News Reader

> Their point about how you can just rewrite relational queries manually and not lose any performance does not seem true the moment that you need to join tables on a condition.

You can do the same thing a database would do - filter the first results on that condition before firing off the second query, or do the join "backwards" if you think that's going to be better / there's an index available for that.

> Their section on reliable replications can be resolved by keeping a log of the changes made to the database rather than a log of the queries.

You'd need to define a representation of that log, which ends up being equivalent to writing a NoSQL datastore.

> I don't understand how their NoSql model of "Download the latest doc during a change" is any better. It basically becomes "last writer wins" in a distributed system.

You at least get consistency. And NoSQL gives you the option of building something better like e.g. Riak does.

> The downgrading to NoSql argument makes little sense to me as well. If your frontend is sqlite then you can have Postgres, or MariaDb in your backend. You need to account for having different queries on the front and the back, but the backend is presumably also operating under very different constraints than the frontend.

This is kind of the same as the log replication problem - you want the protocol for what's replicated from frontend to backend to be something simple that you could implement on top of approximately any backend datastore. "SQL queries" aren't that, whereas a simple K/V store protocol could be (e.g. MySQL actually implements the memcached query protocol, or did at one point - that would be very hard to do the other way around).

> I'd actually make an argument that NoSql databases are a type of relational database heavily optimized towards not needing to do joins

Um, WTF? Can you flesh this out at all?

> Um, WTF? Can you flesh this out at all?

Ha, sure.

Let's say that you have a very-basic NoSql schema for a chat app:

chat_app/person/<person id>/msgs/<chat documents>

so for each <person id> you stick chat messages in the <chat documents> separated by date or something. Conversations with different people end up in different <person id> paths. Chats have metadata like timestamps in addition to the message content

You can also represent this as a SQL table with something like:

Person_Id (uint) | Timestamp (timestamp) | Message (string)

Now the classic argument for NoSql is that NoSql is the best representation for this since you don't really care about doing complicated queries over the chats. You just want to be able to index in and get a persons chats. Maybe sort by the most recent chats so the <chat documents> have the timestamp in the key. That's what I mean.

Where I get frustrated with NoSql is that you inevitably end up needing to redo a lot of SQL logic anyways. A basic example is mapping the person_id -> visible name. The NoSql way is to have a meta file under <person id>, like chat_app/person/<person id>/<meta file> where you would have the user name under there. Then you can look up easily what to display when showing chats for <person id>.

But then you add group chats. Now you need to also have a group chat section and you need to go back to the person_id path to resolve the visible names. Having a relational table with:

Person Id | Visible Name

seems like a more elegant solution to me. Once you need to start gathering stats on the data as well, relational tables become more easy to gather information on how often you are talking to various people or lengths of conversations with people. Most of the functionality for statistical gathering and other use cases can be replicated in NoSql but it involves extra effort that SQL just gives you.

Sometimes data is relational, sure. But even in SQL-land you generally end up wanting to denormalize it for performance. I guess "optimized for not doing joins" is sort of true, but even on an SQL database that's something you optimize for in your live dataflow. Meanwhile not being able to have collection columns or sum types is a much bigger and more common data modelling problem in my experience - yes PostgreSQL has JSON columns but they're non-standard, not supported by all the drivers, not compatible with your in-memory test database, and ultimately still fairly clunky.

> Once you need to start gathering stats on the data as well, relational tables become more easy to gather information on how often you are talking to various people or lengths of conversations with people. Most of the functionality for statistical gathering and other use cases can be replicated in NoSql but it involves extra effort that SQL just gives you.

Now this I do sort of agree with - the one thing SQL databases are good at is semi-ad-hoc statistics/reporting/aggregation. Several systems I've worked on had an ETL pipeline that ended with dumping everything in an SQL or SQL-like datastore for the data folks to play around with. But you can't do that kind of querying in your primary live datastore once you have any sort of scale - at best you'll slow down your main system, quite possible you'll deadlock or worse. So even when I've worked on systems that were all-SQL, we ended up with much the same architecture of an ETL pipeline dumping data out of the primary datastore into a separate reporting datastore.

> > Their section on reliable replications can be resolved by keeping a log of the changes made to the database rather than a log of the queries.

> You'd need to define a representation of that log, which ends up being equivalent to writing a NoSQL datastore.

I think this covered by relational DBs redo/rollback/write-ahead log; whatever you want to call it.

Right, but they generally keep it as a hidden implementation detail, and when you do get access to it it's not in any standardised way. For a lot of use cases you're better off unbundling it and newer datastores tend to be better at that.

Hybrid logical clocks are distinct from Lamport clocks in that they include a notion of local timestamp while also preserving partial order. They are well-established and used widely outside RxDB - see e.g. https://sergeiturukin.com/2017/06/26/hybrid-logical-clocks.h...

> Their section on reliable replications can be resolved by keeping a log of the changes made to the database rather than a log of the queries

I do love Debezium for easily streaming changelogs to Kafka.