Comment by lobster_johnson | Hacker News Reader

Comment by lobster_johnson | Hacker News Reader

Parent Full thread

lobster_johnson

On its own, the code representation is not an improvement. Where it shines is when you want to compose result sets.

For example, in ActiveRecord (the Ruby library that this looks very much influenced by), given:

  posts = Post.where(user_id: 1)

...you can now do things like:

  recent_posts = posts.order(created_at: :desc).limit(100)

or

  tagged = recent_posts.where('tag in ?', ['hacker', 'news'])

and then you can extract data:

  tagged.group_by('tag')

or do mutate it:

  tagged.update_attributes({author_id: 2})

or

  tagged.destroy_all

...And so on. By encapsulating a result set as something that can be augmented with new query clauses (where, limit, select) and so on, you can incrementally build queries, pass them to other functions, store them as member variables so that they be reused as "scopes" across multiple calls, and so on.

If the query were just a string, this sort of thing becomes awkward, verbose, brittle, and generally not type-safe.

The problem with this approach is that you are forced to work in a subset of SQL due to the type system mismatch of SQL and the host language.

A "proper" solution to this problem requires language changes (as Microsoft did with C# 3.0).

You should take a closer look at what Diesel is doing with it's query builder. We actually do attempt to support the full power of every supported backend (of course there's still holes which need to be filled), with proper type guarantees which map to the semantics of Postgres. The main difference being that Diesel can check your queries for correctness at compile time.

The problem is that joins and projections give rise to new record types. The only way to support this in a type safe and natural way is if you have anonymous records in the host language, which Rust doesn't. On top of this there is some other metaprogramming stuff that is necessary in order to facilitate the DSL embedding.

Look at what Microsoft did with LINQ. You can't pull that off in Rust no matter how much you want to.

This can be accomplished with procedural macros.

This can be accomplished without procedural macros, too. ;)

Nevertheless, the goal here is to build an ORM for Rust. Why waste time describing what you think it can't do?

While it's true that solving this at the language level would be even better, it's not required. All you need is an extensible AST data model.

For example, ActiveRecord doesn't have support for window functions, but its underlying AST (a library called ARel) allows you to extend it, which at one point I did, allowing me to write something like this:

  things = Arel::Table.new('things')
  things.project(Arel::Nodes::NamedFunction.new(:sum, [things.count]).over('my_window')).
    window('my_window').
      order(things[:name]).
      range(Arel::Nodes::Preceding.new)

which generates something like:

  select sum(count(*)) over my_window
  from things
  window my_window as (order by name rows unbounded preceding)

(It's been a while since I looked at this particular code, which is no longer in production use, so it might not be correct.)

In other words, I could continue to express whatever I wanted of SQL, at the language level — no type mismatches.

The SQL builder code isn't pretty, to be sure, but it's something you typically encapsulate into a lower-level module rather than writing in application code. And as it's highly analogous to the SQL code it generates, it's not like it's that much more complicated. (ARel isn't a shining example of how to do it right, either. I expect Diesel implements the AST aspect more cleanly.)

Edit: I see what you're saying about typed record types. I was thinking more about the grammar, not the shape of the data.

Yes I should have been more clear that I was talking about a static typing context. In a dynamic language the problem is much easier, as you are basically only limited by your languages metaprogramming facilities.

But isn't the same true for a statically typed language with sufficiently powerful macros? I don't know if Rust falls into that category, having never used it, but I don't think it makes a difference whether the metaprogramming happens during compile time or runtime.

Yeah, if they're talking about what I think they're talking about, yes, something like LINQ could be implemented with macros (compiler plugin variant, I think) in Rust.

> basically only limited by your languages metaprogramming facilities

I'd like to point out that strong static-typing does not preclude powerful metaprogramming! Nim's macro system is extremely powerful because of it's great interplay between it and the type system.

http://nim-lang.org/docs/manual.html#macros

SQLAlchemy is a Python ORM that matches SQL almost 1:1.

The main reasons I like this approach are:

* it makes way easier (and safer) to reuse query parts compared to string composition/interpolation

* the ORM takes care of differences between the backends, so it is easier if I have to port the application to another RDBMS.

That's normally done in SQL with JOIN, or in some cases, temporary tables.

The ORM model of building a query dynamically over multiple steps in multiple contexts gives you a lot more flexibility and more resilient code. You can share code that builds queries for multiple tables that share some, but not all, fields or relation patterns much more easily. And you can do it all without making multiple calls to the database.

See Sandi Metz' recent piece, The Wrong Abstraction [1]. I prefer a SQL template approach such as Yesql [2]. The trouble with an ORM like ActiveRecord is a lack of control of when and how the query is performed.

1. http://www.sandimetz.com/blog/2016/1/20/the-wrong-abstractio...

2. https://github.com/krisajenkins/yesql

That's a problem with ActiveRecord and not with ORMs in general. Plenty of ORMs are explicit about how and when queries are performed. Check out SQLAlchemy for an example.

Reply on news.ycombinator.com