Hacker News Reader

Comment by s_kilk | Hacker News Reader

> Let's start with the simplest one. Making the developer interface to the database a structured format instead of a textual query language was a clear win.

I think this is the most significant factor, by far. With Mongo it's turtles (or at least Maps/Hashes) all the way down, without a strange pseudo-english layer near the bottom that forces you to translate back and forth. For some devs that's a big deal.

For the last while I've been experimenting with bringing the same feature to PostgreSQL (http://bedquiltdb.github.io), turns out it's very do-able, but I don't have enough time to make it as featureful as it needs to be.

View on HN

Maybe I'm weird and just like SQL but the query-as-data aspect of Mongo is actually my least favorite aspect of using it. Ad-hoc queries become torture, as do complicated queries. I might just be lucky to have used SQL Server and C# for so long which eases a lot of the pains you described.

Same. I wish there was a DSL layer on top.

If you use Scala and mongodb, I highly recommend this DSL:

https://github.com/foursquare/rogue

Typesafe queries using SQL-like syntax. Particularly when used with the Spindle library (https://github.com/foursquare/spindle), this is best ORM I've ever used.

That does look very nice. I've never actually used Scala seriously, but coincidentally have been learning how to use it all this week. I'm even more interested, now.

It's pretty easy to build wrappers for things like that, though. Build a constructor that has instances methods of just doing `myQueryInstance.addEqualsQuery(key, value)`. You can then just pass it around and add to the query instance as it moves around, deconstructing it at your database access point.

SQL is still one of the most readable languages in my opinion. Its the one language where I find it easier to read queries than write them.

I should clarify, I'm a big fan of SQL and relational DBs, but it's hard to deny that some/many devs find the abstraction boundary a bit weird.

And really, when you think about it, it is a bit weird, (we send sentences of almost-english toward the DB, then get structured data back) and there's nothing wrong with that.

But many devs do value the ability to stay in hash-map land all day without needing to think about how to cross an abstraction boundary into the DB, and MongoDB actually has a pretty cool solution there.

SQL is fine, no problem with it unless you are embedding queries in a OO application and you are forced to hack mapping layer between the two interfaces, which gives you two options, either compromise performance for the sake of development easiness or vice versa.

I’ve played a bit with Jeremy Evans’s Sequel (http://sequel.jeremyevans.net), and I’m not sure that ORMs are necessarily constrained along this dichotomy (easy to use but with poor performance vs great performance but hard to use). Sequel seems to be about making the hard things easy and reducing the amount of time you need to drop to pure SQL to almost zero.

As someone who has built with Oracle’s Pro C(++) and with various ORMs, Sequel is an ORM that makes me reasonably happy and gives me the expressiveness of embedded SQL without compromising my object model.

> SQL is fine, no problem with it unless you are embedding queries in a OO application

which is essentially every single production deployment of a SQL database in the last twenty years. That's the problem.

A lot of people are using ORM's.

SQL's fine on its own. The problem arises when you try to manipulate it. A query to get messages that were sent between now and a week ago, joining in the user table for both the sender and the receiver to get their full names, is sensible enough when written out as SQL. I can quibble about some of the affordances of SQL, but it's good enough for most queries. The problem is if you try to create some sort of data that represents "get the message body", "messages sent since last week", "get the sender's name", "get the receiver's name", and somehow programmatically assemble a query out of the pieces. (Slight quibble, this isn't actually about OO, this is about "anything that isn't SQL". A Haskell library will have the exact same problem.) An AST-based approach doesn't guarantee that this will be easy, but it's still a step above trying to assemble an SQL query from those pieces.

It's totally possible. There's a ton of libraries that do it in various ways. It's just not even remotely easy, like it should be. If you look into the inside of those libraries you'll find a C'thulian monstronsity of special cases interacting with special cases until the whole thing just explodes into a brain-consuming mess, because SQL was very, very clearly not written for this use case.

In another 10 or 20 years I look forward to data-based analysis that tries to determine how much of the "NoSQL" movement was because the relational data model doesn't work for all use cases, and how much of it was the entirely accidental (in the Brooks sense) problems with A: SQL, the language itself, not its capabilities and B: schema migrations with no essential reason to be as painful as they are. (And something something column stores, but I'm not sure where they fit into this story exactly.) And to be clear on tone, I really am interested. I'm pretty sure the answer won't be either extreme but I'm pretty uncertain about where in the middle we'll fall.

> and somehow programmatically assemble a query out of the pieces.

You can do this today, in various statically typed languages[1] (perhaps dynamic languages as well, though the composition will likely be more ad hoc).

Agreed re: under the hood complexity, but that's more in supporting multiple database engines (and related corner cases) than the transformation of Query > AST > SQL.

[1] https://news.ycombinator.com/item?id=10525040

Having written some of this code myself, I have to disagree; transforming SQL fragments is legitimately frustratingly challenging. I have to admit I don't know how to show a small example that captures the problem, though, because this is one of those cases where the small examples always look easy. It isn't until you're trying to support all of them at once that it is a problem. Combining two fragments that each specify tables, joins, where clause filters, and potentially subqueries with each of those recursively is nontrivial when you get down to it.

It also really doesn't help that "SQL is declarative" is basically a lie, and it very frequently totally matters which "synonymous" query you actually throw at the database, thus eliminating a lot of the obvious clean answers in any practically-useful library.

Definitely not an easy task but the degree of difficulty depends on the language.

Both Haskell and Scala, for example, have sufficiently powerful type systems to allow for building up a typed query expression of arbitrary complexity, which can then be deconstructed via pattern matching to assemble the sql statement. Easy? Not at all, but very much doable, and incredibly elegant...until you need to support various database engines and their limitations/extended features; then the implementation hacks begin :\

Personally I think the work of Stephan Zeiger on the Slick library in Scala is ground breaking. Also, Wadler et al's recent-ish paper on a composable query DSL in F# is worth checking out[1]

[1] http://homepages.inf.ed.ac.uk/wadler/papers/yow/dsl-long.pdf

You seem to persist in believing the problem is on the input side. It's not; it's on the output side. My entire point is that the resulting SQL generation code is what ends up quite hairy, because the way we have found to separate concerns in 2015 and SQL are very, very different.

Let me put it this way... it is precisely because a fluent, Haskell-native SQL querying interface little resembles SQL in either syntax or usage that there is the problem. It is precisely that there have to be these library at all that is the problem. If it were easy, these library wouldn't even exist, or would be little more than drivers, but they're not just drivers... they do a lot of real work.

If SQL didn't stink by the standards of modern separation of concerns, we wouldn't need "ground breaking" work!

Or, to put it another another way:

    $ git clone git@github.com:slick/slick.git
    $ cd slick
    $ cloc .
    ---------------------------------------
    Language   files  blank  comment  code
    ---------------------------------------
    Scala      261    3589   3844     23129

Cut down to just the Scala. 23kilolines of Scala is a lot of Scala! This is not an "easy" task. "Easy" would be something that just wrapped up the existing syntax in a slightly more native form and would clock in somewhere in the several hundred range.

No, I'm saying input is easy, type system does virtually all the work. Output is where the effort is spent (i.e. pattern match on query expression to assemble the statement).

Why these libraries exist is because of string-ly typed programming; in the case of sql: 1) it doesn't compose; 2) is not safe (sql injection attacks); 3) difficult to refactor; 4) untyped, therefore whole class of bugs arise.

And yes, these libraries do a ton of work, well beyond just generating sql statements, which, in the case of Slick pushes the LOC count way higher (non-blocking IO, supports basically every database engine, native function support, jdbc modeled in scala, etc., etc., it's a huge engineering effort, somehow by one person).

Anyway, I'd like to see a much smaller composable query dsl with fewer features and opt-in database support. Compile that to javascript and run in the browser against local database would be very interesting. I think this can be done, but would probably be pretty restrictive in terms of features supported.

I find that SQL is pretty nice in a dynamic language with string templating available... I wrote a simple wrapper for MS-SQL in node using ES6's string templating to convert to parameterized queries[1]. The irony I wrote it while migrating a lot of data out of MS-SQL.. but the interface is probably the nicest SQL client I've worked with (despite being the author).

[1] https://www.npmjs.com/package/mssql-ng

> Making the developer interface to the database a structured format instead of a textual query language was a clear win.

No, it was not. It is an abomiation, just like SharePoint CAML Query or any other "express-your-query-as-an-AST" approach. Sure, it's a paradise for Buzzword-Compliant Fluent Interface Enterprise Query Builder-type libraries.

> Querying SQL data involves constructing strings...

No. Case in point: LINQ. How it is implemented is irrelevant to the statement above.

> ...and counting arguments very carefully

No. Named parameters to the rescue.

LINQ is just an "express your query as an AST" approach. Sure LINQ presents its self as a DSL on top of that AST ( which is exactly what SQL is at the end of the day ), but there are libraries for MongoDB that allow you to do the same thing. There is even a LINQ driver for MongoDB[1]. What you are missing is that by having the lowest common denominator be the AST instead of a DSL you have removed a huge burden from both the server and client side of the operation. From that AST you can build any kind of client side interface without worry about how to parse or compile the result into a specific textual language, even plain old SQL[2] can be used if you like your sever spending its clock cycles parsing text.

1. http://mongodb.github.io/mongo-csharp-driver/2.1/reference/d... 2. https://www.progress.com/connectors/mongodb

I like mongo's interface (most of the time)... but when you get to more interesting queries, it gets far more weird than even SQL... I wrote an early node API almost mirroring the JSON query interface (was easy enough to do filtering on sensitive fields), and it worked really well.

That said, it was really clunky dealing with ordering as JSON doesn't guarantee serialization order (sort fields), and that was cumbersome to work around iirc...

Today RethinkDB would be my first choice for a similar solution... better ops interfaces with replication + failover. Instead of having to do a replica of a sharded system in mongo for distribution with redundancy. Also, while the query interface is a little more complicated to get started with, when your queries get more interesting, it's not nearly as messy as mongo gets.

> or any other "express-your-query-as-an-AST" approach

Well SQL is just an AST abstraction, which was made to be human readable. And that's exactly the issue, SQL is excellent for humans to express querys but really inconvenient to interface with other programs unless you express every single query ad-hoc (which seems the way most people choose since ORM tend to perform poorly)

If you've been programming and using SQL for a few years, it's not a problem. It might take a bit of time to get the feeling of SQL, but it's ok later. And that's fine. rails-15min-blog things are cool and all, but some things have a learning curve and that's ok.

The vast vast majority of SQL in the wild is pretty straightforward and sucks only when the schema sucks (can't blame language for that). Most of the time people hate SQL, they do so because they are working with crap schema (whether someone else created it or they themselves didn't think it through enough). Sane schema + putting a bit of effort write readable SQL can go a long long way.

Given how quickly software engineering as a discipline moves, it's not unreasonable that most users at any given point in time (for any given library) are in those first few years.

Optimizing for the "newbie" case is not a failure.

Absolutely. But optimizing for the long term (and it doesn't seem like SQL is going to disappear any time soon) is not a failure either :)

If a newbie asked me whether to learn MySQL/PostgreSQL, or MongoDB, I'd absolutely say *SQL. Yeah, it's not as straightforward as MongoDB. It's not intuitive for many. But this knowledge will still be useful a year, or 5 years later. Finding a job will be easier too. So, yeah, it could be better, but it's still a good, safe choice.

Am I alone in finding SQL/normalized relational databases to be easier to learn?

In the relational world, you can get a lot of insight into what will and won't work with simple math. There certainly are surprises in every database, but most things are not surprises.

NoSQL databases tend not to leverage such a clear and complete model. You have to study each implementation to learn where the surprises are, and there are more of them.

That's not to say they don't both have their role, I just wonder if people who have a little (and in my care very little) CS theory still remembered find RDBMSes easier than those who have learned development without theory.

> Am I alone in finding SQL/normalized relational databases to be easier to learn?

Not at all. SQL is trivial. Thinking through some joins is tricky, but it's usually because I actually underestimated how logically tricky the thing I was trying to do actually was. Optimizing queries can be very difficult, though.

>If you've been programming and using SQL for a few years, it's not a problem.

Years of SQL injection mitigation don't agree with you. Are you sure you just don't prefer it because you are familiar with it?

It's almost as if constructing SQL queries by concatenating strings from user input is a bad idea.

I don't see how using vanilla mongo in the same fashion would solve the blight you have placed on SQL.

In Mongo, `db.findOne({"username":"$_GET["name"], "password":$_GET["password"]})` won't open your database to the public.

While SQL is a powerful language, the article is right. Mongo query language is useful enough to easily give application developers a place to query and store data without giving them enough rope to hang themselves.

Not a fan of mongo's query format/language, but I have to agree. It's always bugged me that so much work has gone into relational databases, and yet the only way anybody interfaces with them is by using an extremely quirky programming language from the 70's (or whenever).

>>> extremely quirky programming language from the 70's (or whenever)

Because it doesn't have to be shinny and all, it just works. There are no widely used alternatives, so we can say that people are happy-enough using it. See the situation with JS. CoffeeScript and all, it's quite popular. Is there an analogy (in terms of functionality and popularity) in SQL? Nope. No one bothered to do it well enough, or not enough people found that it's worth enough to learn it.

We don't need to reinvent something every few years just because new JS frameworks come out and others are forgotten every few months.

Well you do have MS Access query builder, which I used for a while. Once you got used to it, it was maybe a bit faster than using real SQL. But then it locked you into the query builder way of thinking, and didn't provide very much beneft over textual SQL.

For this dev, it's a big deal. I love writing a query in MongoDB way more than I love writing a query in MySQL.

Granted, I think a big part of it has to do with joins being the annoying part of SQL...

That seems to add to the idea that NoSQL fans just don't know how to use SQL properly.

Are you implying you're not supposed to use joins in SQL?

I think he's implying that finding joins difficult is a sign of immaturity of the developer rather than a problem with the language.

Joins might be tricky if you are unused to them, but they are the secret sauce of relational databases. They are what make it possible to have dozens of views on a single normalized data structure.

This just adds to the idea that sql kind of sucks in terms of usability. For the record I always use sql, but let's be honest with ourselves.

What's annoying about joins? I think practically speaking, whether you're going relational or not, joins are a reality of a data model that supports multiple views and transformations. Assuming that's the case, the only question is whether you're doing joins ad hoc in your application code or you've learned the handful of variations that work natively in your RDBMS.

I actually built a MongoDB query interface for SQL via Python+SQLAlchemy. It allows you to query your relational database just like you would with MongoDB, and in addition use some things (e.g. Joins in queries or "deep querying") that are not possible using MongoDB.

Here's the repo:

https://github.com/adewes/blitzdb

The implementation is stable but I'm still working on finishing Python3 support and documentation.

Reply on news.ycombinator.com