Reasons to love SQLAlchemy
pajhome.org.uk
pajhome.org.uk
So if you are thinking about using SQLAlchemy you should make a conscious decision whether Core or the ORM are better suited for the problem at hand.
What makes it especially amazing is that Mike Bayer has written 90 % of the code base by himself (https://github.com/zzzeek/sqlalchemy/graphs/contributors), which somtimes makes me wonder if he is a real person or just a pseudonym under which a group of talented programmers publish code ;)
Mike is a master.
Do you plan on adding await/async support to mako ?
Cheers
Anytime I write a web frontend in anything other than Python, I miss it very dearly. But really it's well-suited for any kind of templating work, even code generating—I've seen a few projects use it to generate C code. It's very powerful and doesn't try to dumb itself down because of some dogma about not using logic in templates.
session.query(Thing1).join(Thing2).join(Thing3).filter(Thing3.name.like('%easy%'))
I'll be in the middle of firing up psql and then I'll think about writing out my joins and laziness wins out. For more complex stuff I normally go back to SQL, but mostly because I'm less familiar with constructing some of the more complex queries in SQLA (though I'm slowly pushing myself to learn more).In actual code, I'm actually pretty content with raw SQL (wrapped in Perl's DBIx::Simple these days), as I prefer to move the more complicated things to functions/views before constructing elaborate abstractions in-place. Complex ad-hoc queries are another issue, of course...
git checkout --recursive, etc.
Or just a symlink and include it in your setup.py. Or check it out inside your project.
Still allows better organized code. Given, the alternatives are very metaclass heavy black magic, and sometimes spare on comments - actually nearly all ORMs are very heavy on the black magic :)
If you've got wifi/ethernet, configuration through web servers comes naturally.
Even without using the ORM part, the database objects have information to help me roll my own cross-database SQL calls using a bit of string substitution so "?" becomes "%s", etc.
1: http://docs.peewee-orm.com/en/latest/peewee/playhouse.html
I did a bit of research into the Relay stuff but found it hard to find examples that explain the api/protocol clearly. So long as you were using the facebook libs you could follow the examples but I'm more interested in understanding the mechanics – do you have any pointers of where to look.
Also a great place to ask questions is in the GraphQL slack [5]. #python is where the discussion for the python port is happening.
[1] https://facebook.github.io/relay/docs/graphql-relay-specific... [2] https://facebook.github.io/relay/graphql/objectidentificatio... [3] https://facebook.github.io/relay/graphql/connections.htm [4] https://facebook.github.io/relay/graphql/mutations.htm [5] https://graphql-slack.herokuapp.com
It has built in support for SQL::Translator: https://metacpan.org/pod/SQL::Translator which means you can call ->deploy() on just about any database.
For example, in SQLAlchemy there is `column_property`[1] which allows me to do something like:
class User(Model):
id = Column(Integer, primary_key=True)
first_name = Column(String(50))
last_name = Column(String(50))
name = column_property(first_name + " " + last_name)
Querying this model will yield the SQL among the line of: SELECT
user.id,
user.first_name,
user.last_name,
user.first_name || " " || user.last_name AS anon1
FROM users;
This example may not seems very useful (and of course joining a string in the database is not very exciting), but the same pattern could also be used for doing subqueries with SQLAlchemy Core[2], e.g. User.post_count = column_property(
select([func.count()]).where(Post.user_id == User.id)
)
Which yields: SELECT
user.first_name,
user.last_name,
(SELECT COUNT(*) AS count_1 FROM post WHERE post.user_id = user.id) AS anon_1
FROM users;
How SQLAlchemy decouple the object model from the schema and only map them after they are retrieved makes its ORM very flexible and powerful. Ability to easily building SQL is one thing, but how SQLAlchemy is designed to allow me to embrace SQL when necessary rather than trying to hide everything away from me makes me really love it.[1]: http://docs.sqlalchemy.org/en/rel_1_0/orm/mapping_columns.ht...
[2]: http://docs.sqlalchemy.org/en/rel_1_0/core/tutorial.html
Thanks!
1. sqlacodegen (https://github.com/ksindi/sqlacodegen) - Creates a single Python file with the SQLAlchemy model from your existing database.
2. Bulk inserts:
new_people = []
for person in list_of_new_people:
new_people.append({'name' : person.name, 'age' : person.age })
People.__table__.insert().values(new_people)I've prototyped databases in under 30 minutes using the combination of:
1. MySQL Workbench for design and database creation
2. sqlacodgen to create SQLAlchemy Python model
3. SQLAlchemy's bulk insert to populate the database
I recently created a new API and went for a very simple flask.Methodview along with Marshmallow [0] for (de)serialization. I then wrote a simple replacement for marshmallow-alchemy since it wouldn't do object loading without a lot of flapping about.
I found Marshmallow really nice to work with (I'd been using Schematics in the past).
I've been kicking around different approaches to this for over a decade, going back to Castor (Java data binding), and with Python, SQLAlchemy, and the libraries mentioned here I'm finally feeling optimistic.
Request parsing in Flask-RESTful seems bolted on, and the fields in WTForms seem frustratingly redundant given that almost everything I do is going through the API anyway. Maybe focusing more on the Fields themselves will help tie it all together.
I actually like Django's querysets a little better than SQLAlchemy's, but most other things better in Flask. I may be a heretic.
The declarative API in SQLAlchemy requires much more work to set up relations, and the Django way of being able to chain relations in a query with "__" is pretty friendly in ways that I like.
(joking, obvs)
It's always interesting to rummage cross platform to look at libraries so you can steal the good bits. SQLAlchemy has its roots in Hibernate for example.
I'm also a very happy user of Alembic. I change my models as required and then use autogenerate to create the migration scripts. Then I tweak to account for slightly more esoteric things and data migrations. I find that it gets me 80% of the way there without having to think too much about the data model changes while I'm doing the initial development.
Two developers create a migration on the same day, but they get merged later. Alembic stops you and ask which goes first. ActiveRecord::Migration runs them both. This is particularly bad when one migration is a table drop and another is an add_column on the same table.
And that's just one short coming. Our monkey patched ActiveRecord::Migration doesn't do this (yet), but we've done quite a few other improvements to it.
Another significant difference is the migration generation. Once you've generated migrations with alembic, it's hard to tolerate anything else.
.join.filter.like.order_by etc.. is crazy imo. It's often more complicated than that and doing it in a non-sql way is suboptimal/unmaintainable.
Use ORM for: 1. INSERT, UPDATE, DETELE get one record
Use SQL/Stored Procedure for: SELECTs that involved more complex joins and queries.
However, in the select, where'd you'd have to join all three tables, you might use SQL, yes?
I'm just curious how people are using ORMs vs SQL, so sorry for the questions.
If you're using a different ORM then things are probably going to be different. I've worked with a few in various languages and SA is the only one I've found that allows me to express any complex query directly via the ORM. I've heard there are some edge cases where you need to drop down a layer but I haven't run into them yet.
Right now I am working with a suboptimal database design, where I need two nested subqueries to get the most recent event in a related row. I changed them to joins then to (indexed) temporary tables, to try and improve MySQl's performance. It worked to a degree. I doubt that would be possible in an ORM as it is working at the databse level rather than a hgher abstraction.
It's also more composable, if you ask me. I like to think of it like "building" SQL statements using smaller components. Just like you do with code, you essentially compose a complicated business process using smaller blocks.
It's beautiful in it's simplicity. Even if the syntax requires some extra work/learning.
I've created a bunch of base queries that map from my user account table to each different table that users have access too. I can then use these base queries as a starting point to make sure people don't access records that don't belong to them.
bq = db.query(Obj).join(ParentA).join(ParentB).join(User).filter(User.id==bindparam('user_id'))
q = bq.filter(other_rules).params(user_id=1)...
It's a really flexible system.ORM's win hands down over multiple insert and update queries. Complex queries are usually better in SQL.
subqueries are best expressed in sql directly. They can get complicated, especially when using the with clause.
I am not saying that all of what ORM offers is bad. But beside the very basics they should step away. The less they offer the better.
# a subquery
demo_accounts = db.query(Account.id).join(Client).filter(Client.name=='Demo')
# used inside a query
print(db.query(Account.name).filter(Account.id.in_(demo_accounts)))
SELECT account.name AS account_name
FROM account
WHERE account.id IN (SELECT account.id AS account_id
FROM account JOIN client ON client.id = account.client_id
WHERE client.name = :name_1)
# or as a cte
da_as_cte = demo_accounts.cte()
print(db.query(Account.name).join(da_as_cte, da_as_cte.c.id==Account.id))
WITH anon_1 AS
(SELECT account.id AS id
FROM account JOIN client ON client.id = account.client_id
WHERE client.name = :name_1)
SELECT account.name AS account_name
FROM account JOIN anon_1 ON anon_1.id = account.id
There are obviously much more complex cases, but SA tends to handle things in a pretty sane way. You can compose query segments (like above) and if you really need to go back to the raw sql, you can and still have the results mapped into your python objects.Also, you're neglecting all the other benefits you get. Things like the loading strategies and the unit of work tracking are extremely powerful abstractions.
They obviously have great value for CRUD applications however, and they prevent you from making mistakes that others have encountered (and fixed).
That said, I really want to try out the Postgres + SQLAlchemy + Flask + Flask-Restless (as mentioned by others) combo. I usually end up writing my own queries
Also, I'm fully aware of using raw queries in SA/most ORMs -- my point is that if you have to drop down to raw queries, you're paying (mindshare/complexity/concentration/project size/whatever) for an incomplete abstraction you just had to side-step.
Just to make sure there's no hostilities -- I'm not saying "don't use ORMs", because that would be dumb, as they offer tremendous value (and no downsides except for maybe complexity, as you can easily write custom queries) -- I just want to point out that the argument against them still stands, so don't drink too much of the koolaid.
q = db.query(P)
# add this later to fix your issues
# q = q.options(subqueryload('children'))
p = q.filter(P.id==73)
for c in p.children:
# oops, loads of db queries here
pass composite primary keys
complex indexes (functional etc)
arrays + json(these are in django, but I think only recently and before that in contrib)
server side cursors
the non-orm part(the lower layer)
sqlalchemy-alembic (also in django, but only recently I think, still probably less features)
server_default (define a default value for a column that will be applied to the sql-schema and not just model.field)
more customization to the lower level db driver(psycopg2, maybe this is also supported in django)
Use the models + library outside of your web app (ex: in several non-request-serving processes )
There are alot more features that I haven't used/don't know/didn't need.1. Django's built in migrations are essentially South 2.0 and a poster above implied that South was more featureful than Alembic. I couldn't say for sure.
2. "Use the models + library outside of your outside of your web app" - not sure why this can't be done with the Django ORM? I use the ORM for many background and batch tasks
It would be interesting to know What SQLA afficianodos think of the new goodies in Django 1.7/1.8:
https://docs.djangoproject.com/en/1.8/ref/models/conditional...
https://docs.djangoproject.com/en/1.8/releases/1.8/#query-ex...
https://docs.djangoproject.com/en/1.8/releases/1.7/#custom-l...
Obviously the main reason to love the Django ORM is it's tight integration with Django but I think nowadays it's rather undeserving of it's "SQL Alchemy's poor cousin" reputation.
Both let you escape into bare SQL easily, and Django ORM integrates well with Django, what SQLAlchemy doesn't.
I have wrestled the Django ORM into doing some relativley complex queries, but looking back, I am not sure it is worth the effort. The code is more likely to be harder to undersatand by anyone except me, and I started with the equivalent SQL and tweaked the ORM version of the query until it worked. You still need to know the SQL that will be produced when you start using the extra clause.
Maybe SQL alchemy is better. I guess I'll find out soon as my new workplace is using it.
Easy things are easier in Django. Harder things are easier (or even possible) in SQLA.
But that's not quite accurate either, it's more a case of Django being a bit less work to get started with while SQLA needs a little more effort up front. But once set up and mapped, SQLA isn't any harder to use than Django.
I'm a huge fan of this library!
I'm building an app to facilitate more code giving to nonprofits, and help nonprofits move to the open source world. It will match GitHub coders to nonprofit projects based on skills, interests and other things. It will provide an interface for less-technical people at nonprofits, many of which can't afford tech salaries, to communicate needs and requirements. It will do as much as possible via GetHub (e.g. Pull Requests) so for coders there will be no added friction over the already low friction giving GitHub enables.
It's a pretty thin WSGI wrapper around a bunch of best-in-breed Python libraries including SQLAlchemy.
SQLAlchemy + Python (preferably using Flask) and the Mega Tutorial by Miguel Grinberg is the best way to get started in web development. I realize every body has different methods that work for them, but not only did this work for me, it also was very well explained by Miguel and also as OP said, the documentation on SQLAlchemy.
It was sad to bid adieu to SQLAlchemy recently, as I picked up MongoDB (you should check out MongoEngine if you haven't!)
Since the ORM wasn't forced on me, I was able to come around to it on my own. I've started to warm up to it recently even though I can't say I'm totally sold on it. The nice thing I can say about the ORM as of right now is that I don't feel like I'd be throwing away my core experiences to embrace it.