Django and Postgres for the Busy Rails Developer
andyatkinson.com
andyatkinson.com
The absolute worst Django feature is the templating language. It seems to be designed to slow down developers to the like of old time Java web apps, almost mandatory templatetags et all.
The query language is moderately bad, quite verbose (Model.objects every time) for no good reason.
The lack of common project structure means that every project is different.
There is no Capistrano to deploy. I wrote something like that myself and we have been using it for maybe 7 years.
I'm sure I could go on for a while if I keep thinking about it, but you got the gist of it.
On the Rails side, sometimes I'd like to have a talk with some of the previous developers of the Rails app, which hid some important functionality in a before save callback in a different module for no particular reason, but one can be too clever with Python too. However the language (Python) is quite dull, which can be a good or a bad thing. It's very subjective. It's a Ruby gone bad at design time to me.
Do you think Rails is still worth using when investing in learning Python and Django has a much higher roi?
I just never switched to Aptana or Rubymine and now I don't wanna.
Personally I like the ruby-lsp and TPope’s Rails plugin in NeoVim.
I dislike heavy weight IDEs, I never find the juice is worth the squeeze.
On the plus side, ruby-lsp is always being improved and worked on, so just like Ruby in general it's only getting better at what it does.
Once you get past the beginner level in Django, you're going to pick up a ton of Python knowledge (standard dunder methods, MRO, standard lib, data type's im/mutability, package ecosystem, etc.) and muscle memory along the way. Django (for the most part) is just plain old Python data structures, classes, and functions that a decent Django dev will apart, reuse, override, repurpose, and add on to as they do more interesting things. Python is boring (in a good way), however it has a lot of surface area (std lib is massive) and intricacies that a Django dev will pick up along the way to becoming an intermediate/advanced dev. It would take someone coming in fresh to Python quite a while to catch up. Naturally, if the same person knew several languages, the ramp up would be quite a bit quicker.
Rails has been around two decades, Ruby three. Both will still be there down the road when you decide to try it out. A lot of what you learn in your Python work will translate in some way to Ruby & Rails.
I’d also contest your statement that learning Python and Django have a “much higher ROI” - but, I don’t actually want to get into it.
> Rails is still worth using
Yes. Hotwire + Hotwire Native are great tech. The whole suite is integrated in a way that Django isn't ime. I prefer the strong default approach for most things.
Seems to be... But it can be used for anything Python is. There's ML libraries, bindings to stuff like Torch, all the math-y stuff like Python, all sorts of stuff. Also it's a great language to just roll your own scripts, programs, etc... Then there's also mruby, which can be embedded like Lua...
My ideal enhancement to Django would be something like how Microsoft made Razor into Blazor... A template engine that can run purely on the back-end or purely on the front-end, replacing any need to ever use JavaScript, you stick to your native programming tongue if you will.
[1] https://pep-previews--4124.org.readthedocs.build/pep-0750/
[2] https://htpy.dev
The nice thing about Blazor is you can either have your templates as WASM, or have them be back-end driven, and Blazor does a bit of the gluing for you so your site updates as things happen and need to be updated, like Phoenix LiveView.
The problem with templates in general is you can write malformed HTML in them. The nice thing about htpy et al is you simply can't do that. I feel like what we really want is JSX in Python, where you can write XML-like syntax and it's converted into Python at import time.
In the end, my instinct is that t-strings are the better more generic feature to add to the language itself today.
That said, I'd love to see the Python ecosystem get to the point where it's relatively easy to implement transpilers and get new grammars integrated with key tools (colorizers, formatters, linters, type checkers, etc). After that: let a thousand JSX-likes bloom.
Looks like there's some attempts for Phoenix Liveview for Python.
https://github.com/liveviews/liveviews?tab=readme-ov-file#py...
But the ecosystem of React, including UI libraries, charting, forms, and so forth, is enormous.
And you can add a DB and let there be managed by some other system.
Same goes for extending manage.py
I’ve worked on Django projects big and small and generally the speed are which we could deliver even on strange edge cases impressed.
The Django ORM is loved by tens, maybe hundreds of thousands of Python devs globally, and while I could raise concerns about it (for example, the async experimental capabilities fucking suck, and the core dev team is extremely slow to innovate on them on the basis that that's not "the Django way"), I've never heard anyone but a Rails dev complain that it's "too verbose".
People love it so much that they'll do atrocities like import it into Jupyter notebooks or simple scripts that should otherwise have no business in bringing in those type of dependencies. I should know because when I've felt creative and a bit naughty about doing those types of things I've found out - to my unmeasurable disappointment - that there were many other trailblazers before me to write gists, Medium articles, and almost everything every type of format short of an entire O'Reilly book on how to do it.
Maybe not too verbose, but there's a large population of python devs that find SQLAlchemy (+ Alembic) the much better option.
Django's ORM is miserable.
User.objects.filter(company__product__product_variation__var_type__in=['var1', 'var2'])
vs
User.where(company: { product: { product_variation: { var_type: ['var1', 'var2'] }}})
Little things like that make it so much harder to read.
With SQLAlchemy it's easy to build mappers to real domain objects that are independent of the database. Most Django people treat the Django models as entities or, worse, do business logic in views etc. It's possible to implement unit of work on top of Django, but that's then another layer you have to maintain yourself (which SQLAlchemy does for you).
But SQLAlchemy is harder. It's better but it's harder. Django is easier to get going with. But it bites you later (unless your app is just CRUD, in which case Django is all you need, well that and a better way to generate HTML).
It’s a powerful idea but having helped people with it I’ve mostly felt it reinforce my belief that less magic is usually better from a support perspective.
You would rather inline them into a single nested magical variable name instead?
One of the problems I find is gradually complexity creep from the business. It starts off as a little clean method here, a signal there etc. Before you know it you've got a domain model tightly coupled to CRUD primitives.
The much higher risk IMO is building a hyper-flexible application from the start knowing what you need today is a CRUD app, just in case in future you need something else. That's how you get shelfware and awkward conversations.
With the addition of Phlex::Kit, it has made building out a component library pretty easy too. RubyUI https://github.com/ruby-ui/ruby_ui does a great job of showing off how to do this.
I keep meaning to take a look at Phlex.
Furthermore your can't copy and paste examples. Think about Bootstrap components. You have to really write everything from scratch.
Rails has nice things, but overall, I prefer Django's approach and the language it uses.
I think my take is that there's no middle-ground between Rails and some of the newer Java-based frameworks.
Either I'm doing a side project or startup, and I need to go as FAST as possible, type systems and similar be damned, or I'm OK with not going as fast as possible and we're going to go slower but deliver something very solid.
Python/django still had me spending a bunch of time on stuff that rails will either generate, scaffold, or hide from you completely. While still not being nearly as safe or performant as anything in the Java world. Note I'm mostly thinking about more modern frameworks like Javalin, not necessarily Spring's entire ecosystem/way-of-life.
Django's main benefits are
1. Admin interface 2. Python
Having an out-of-the-box admin interface means business people can operationalize and workaround short comings of the software today, not tomorrow. I think functional admin interfaces can often be the difference between a successful company and one which is constantly operating off of spreadsheets in the background and never fully commits to their software.
Switching between Django projects while doing contract work was always a "what the hell?" experience. Rails' opinionated nature makes it easy to dive into new projects you know virtually nothing about.
Nowadays, I focus exclusively on Elixir/Phoenix apps. Elixir eliminates the OOP requirement (I've never liked OOP because I don't feel it remotely lives up to the hype) while maintaining all the other great things I love about Rails, like consistent project layouts. It also makes writing multi-threaded, distributed apps easy.
Unless, you are more angling at the project vs app distinction? Which does seem a bit of cruft. I only ever create a “core” app and put everything in there. Avoids any potential headache.
My only really basis of comparison would be Flask, which has basically zero standard. You can organize your app however you want. Given the barebones nature and need for third party dependencies, no two Flask apps will look alike.
I agree with this, but the Django tutorial would lead someone to believe that you need a bunch of apps. So I've seen newbies create way too many apps without understanding why they've done it.
It swaps the templating engine for React. But still server side and using all the Django features you know and love. No SPA needed.
In theory, you could swap in Svelte in there.
They wanted to provide safe and clean templating, so they decided to force developers to write tags instead of putting logic inside templates.
I agree it's quite overkill because I still want to run code like I would with ERB or PHP sometimes instead of being forced to write a tag. But this can lead to difficult to debug templates and security problems.
> The query language is moderately bad, quite verbose (Model.objects every time) for no good reason.
The idea was to make the separation between Model and QuerySet objects obvious. When you access the objects property on the model object, you're actually accessing the QuerySet.
It seems weird because in Rails, everything is encapsulated in the model's class. But Python devs tend to prefer explicit code. In Django we also have the Manager class as an additional layer used to build QuerySets. I'd say it's just a different architecture instead of a shortcoming.
> The lack of common project structure means that every project is different.
This is a real problem but I reckon that big Rails projects aren't as easy to navigate either because most don't stick to the Rails way as soon as custom business logic is needed.
I would say that Django isn't worse than Rails. They just decided to be more strict in some places, which isn't necessarily a bad thing. In the end, you just need to work with whatever you prefer.
Made a mistake there. I wanted to say: The objects property contains a Manager instance that allows you to work with QuerySets.
I can see how having three layers to work with your database can cause confusion.
> The idea was to make the separation between Model and QuerySet objects obvious. When you access the objects property on the model object, you're actually accessing the QuerySet.
> It seems weird because in Rails, everything is encapsulated in the model's class. But Python devs tend to prefer explicit code. In Django we also have the Manager class as an additional layer used to build QuerySets. I'd say it's just a different architecture instead of a shortcoming.
Also, you're supposed to enhance the models with additional functionality independent of views. Not just in the form of just adding new class or instance methods, but you can also redefine or add alternate QuerySet Managers. For example "Model.objects" could be replaced with a pre-filtered one that includes "deleted=False" and you can add "Model.deleted" for "deleted=True" and "Model.all" for the original get-everything QuerySet.
https://docs.djangoproject.com/en/5.1/topics/db/managers/#cu...
I will say one thing in response: we've found it approximately 10x easier to find devs having years of experience with Python than with Ruby. That's a non-trivial argument in Django's favour in our organisation.
I understand the argument for things being "explicit" in Python, but then I just prefer Java if I'm going to be very verbose about what I want (I'm only talking about backend APIs here). It's just my opinion that whether explicit is good or not depends on the level of abstraction we're interested in, and I don't believe that explicit is always good. (I like the concept of meta-algorithms, for example)
But if there's a one-person project that needs to be scaled quickly, I prefer Rails. (The article mentions how Django makes model fields explicit in the models file, but doesn't talk about schema.rb in Rails which doesn't require you to view each migration to know how the database looks.)
Yes, big projects in any language can get messy, but that's a software engineering problem, not a framework problem.
I recently wrote a FastAPI project that was db-driven, with all the necessary test cases, etc. The amount of lines it took to express the controller, the schemas and models separately, the dependencies for auth and stuff, and especially elaborate test cases was pretty substantial. Yeah, the code was all explicit, but it was not enjoyable.
Strongly disagree. IME the biggest factor determining how big your project can get before it turns into a mess is your choice of language/framework.
I’ll never understand the conventions of Python land, but at least the language is flexible enough to do it properly.
Python is fairly accepting with modules being either directories or files.
Going the dir/__init__.py route also allows you to do a bit of encapsulation as you can not export things.
Of course, someone could import from your file directly still. But, yeah, there is no reason to keep all of your models in a single file.
Zulip in general is a great example of a large open source Django app that's been maintained and actively developed for a long time. I use it as a reference quite a lot.
Python has the best machine learning and AI support too. It is very easy to mix geographic libs with data science tools.
To build a simple CRUD software with vanilla SQL you really don't need Django and Python. But I challenge any one to try a GIS full stack without Django and Python.
Just look for GDAL, pyproj, shapelly, fiona and MapLibre/Mapbox.
And it really powerful to combine with HTMX or Vuejs for reactive SPAs
Once the data is loaded into memory, everything else is just syntax.
At this point, I only reach for python as a web application if I need to build something that has AI / ML features. Rails apps I've built in the past that needed to call a local model resulted in having to create a python inference microservice, which kinda sucks.
Also, Jupyter and Zeppelin notebooks allow for interoperability, although I don’t know how.
(Oh, lastly, if there’s a JVM implementation of Python you could probably use alongside JRuby)
The stuff I looked at ages ago was for literally embedding Python code in Ruby code. I never used any of it either, so I don’t have any recommendations.
Edit: Actually, rather than have Python connect to the Rails DB, have Rails also connect to the Python DB (note that this can be on the same db server, etc). Rails can then handle all the “I’m a product” stuff (user accounts, etc) and then read the Python data for the rest. Would that work?
For me, if I was doing minimalist CRUD and wanted to ship to mobile or a lot of users, I’d reach for Rails. Hotwire and Turbo seem great, and the ecosystem just seems better and more developed for customer facing stuff.
Most of my work has been internal tooling with an emphasis on data analytics or geospatial. Having python libs run natively is a big win for data analysis, and as another commenter posted, geodjango is very tough to beat for usability for geospatial work. For this I still maintain Django is pretty unbeatable.
I am happy to be corrected in both domains though, again, not much rails experience here beyond a couple toy projects.
Also, I’ve worked with some pretty gnarly pure-SQL systems that had essentially the same kinds of problems.
It’s all just variations on “saving hours of planning with months of work” kind of things.
Hmm. Thinking over my career...
Maybe I don't know what you mean by "domain objects", "separation", and "persistence", but when I think over the stuff I've worked on in my career, and the problems I've seen in codebases...
Yeah, I think I'm with them on this - your domain objects and your persistence should pretty much be the same thing. There's gonna be a few exceptions (and, ofc, you can do a shit job building the models), but I expect those to be short-lived and small.
Otherwise, you shouldn't have domain _objects_ doing all that, you should have functions operating on data, and the data is pretty much always going to be either (1) some incoming hash or (2) some database record; then you group the functions into handy modules for humans. And then they're not domain _objects_.
Do you have a good example of a "domain object" that you think really _should_ be heavily separated from persistence?
PS - The one time that comes to mind that might fit, the data I wanted to persist was essentially logging data on the operation that was performed on other data.
PPS - N+1 queries are kinda trivial to resolve in most cases in Rails? There's core ORM functionality for it.
PPPS - Yeah, don't put your confirmation mail logic in the model callback, that's a bad plan.
Fake edit: Like, you either have data you want to persist, or you have data you're transforming. If you're transforming it, it's temporary, and either I don't understand what a "domain object" is, or I think you'd make a mistake to make your temporary data into a domain object.
By "domain objects" I don't mean that you need to embrace OOP, they could be simple structs without any associated logic.
> PPS - N+1 queries are kinda trivial to resolve in most cases in Rails? There's core ORM functionality for it.
The problem isn't that you can't fix N+1 queries in Rails, it's that it's incredibly easy to create them by accident because your ActiveRecord objects carry a DB connection around everywhere, even in views.
And yes, Rails makes it easy to make "unthinking" mistakes - like, you should prep your ActiveRecord objects in the controller so they've got all their data for the view, but, because you don't have to...
Blessing and a curse. You can turn it on and pretty much immediately get started, but that doesn't mean you know what you're doing...
Though in a world of SQLC (and similar) I’m not sure what ORMs are really solving for anyone anymore.
My explanation of Django's approach to migrations would involve a lot more expletives. It is by far my least favorite thing about the framework.
- Fields are not-null by default, the opposite of SQL itself
- Field declarations use argument names that sound like the SQL equivalents (default, on delete cascade/restrict), but they're fully enforced in python and don't make it into the SQL schema at all by default. I get that not every DB supports these features, and there are sometimes good reasons for doing something like a delete cascade in process (if you need to implement custom on-delete logic or something), but the DSL shouldn't read like SQL if it's not going to change the SQL schema. The default value thing combined with not-null by default is particularly easy to get bitten by adding a new field with default if you deploy migrations separately from deploying code (which you must do to avoid downtime): if you add a new field with a default value believing it's safe, you will probably get crashes in the interval between applying the migration & the new code deploying because you've got not-null column without a default value in the schema. They did finally add db_default recently, thankfully, but it took years!
- Django migrations cannot be understood in isolation, the sql a generated migration containing something like an AlterField operation will run depends on what earlier migrations say. You have to check with the sqlmigrate command and/or actually read earlier migrations to be sure you understand what the migration will do. Compared to Rails, where each migration can be read and understood in isolation (though you may still need to understand how a Rails migration DSL will translate to actual SQL of course). This also has a performance impact making Django migrations slower because Django has an in-memory model of what the schema should be at each migration point, so running a migration is not just "load the file and run the commands", it's "determine the schema that should exist after running this migration, diff that with the schema we think exists before this point, magically generate SQL for that diff, then run that".
- The makemigrations command to auto-generate pending migrations is very aggressive and will detect things as "changed" that don't impact the schema. If you changed some help text on a field, or a localization string, or the aforementioned only-in-python default value, makemigrations will see that as requiring a migration that does nothing. Leads to lots of cruft.
- Related to both of the above points, AlterField's auto-generated SQL can be dangerous and bad. Particularly, I've seen cases where very minor changes to a ForeignKey (like changing from nullable to not-nullable, or even not-schema-impacting changes like above) would, by default, have dropped the foreign key constraint & index, and then re-created them. Completely unnecessary and potentially dangerous since it could be locking a large table. I'm not positive, but in some cases I think these have been generated purely because of a django upgrade leading to it deciding the names of the indexes/constraints need to be changed for some reason.
- AlterField will also tend to stomp all over any tweaks you manually made to the schema to work around all these issues. If you manually wrote a SQL statement in an earlier migration to add a default value to a column, and then that column's definition gets tweaked months or years later the generated AlterField is gonna remove your default value. At a technical level this isn't surprising when you understand how Django is modeling the schema internally & generating the SQL changes, but it's definitely a bad user experience downstream of a lot of these design decisions.
Generally the field declaration/migrations system in Django feels to me designed to lead people down a garden path towards bad and dangerous behavior. If I had my druthers I'd enforce a policy in our Django app of "never run makemigrations, all migrations must be manually written SQL".
It is interesting... I strongly feel the opposite after debugging far too many SQL queries where someone checked for an empty string but forgot to check for NULL. NULL by default is a SQL footgun IMO.
The point about Django’s choices about default and not-null though is that it can easily lead to crashes while you’re adding fields. If you add a string field with default=“” and don’t specify null=False, the generated schema will be a non-null field without a default value in SQL, but Django will backfill “” into all existing rows. To avoid downtime, you need to deploy migrations & apply them, then deploy the models.py/other code changes. But if anyone tries to write a new row before the new code finishes deploying after applying the migration, it will crash.
`makemigrations --dry-run` is helpful to see if your manual migrations are complete but besides that I agree
This behavior is why `SeparateDatabaseAndState` is a necessary hack in Django: sometimes you need to do an `AlterField` where the SQL Django would generate is really bad, so you need to write your own `RunSQL` to do the right thing, but you also need Django to see the `AlterField` as applied or you'll have problems with future migrations.
I suppose I could modify my preference to "run makemigrations and then wrap every single op in SeparateDatabaseAndState", but that does not sound fun :).
# Attributes that don't affect a column definition.
# These attributes are ignored when altering the field.
non_db_attrs = (
"blank",
"choices",
"db_column",
"editable",
"error_messages",
"help_text",
"limit_choices_to",
# Database-level options are not supported, see #21961.
"on_delete",
"related_name",
"related_query_name",
"validators",
"verbose_name",
)
I often do `on_delete=models.DO_NOTHING, db_constraint=False` and add the on delete cascade constraint manually to the migration file. https://stackoverflow.com/a/78668897/288424
Same goes for `default. Prior to `db_default` I'd often edit the migration file and set the default in SQL.
I much prefer Elixir's Ecto now.
IMHO the major upsides of Rails were subsequently adopted by almost all major frameworks in other languages. It may not be sexy or make HN front page a lot, but Spring Boot definitely "caught on", for example.
- ORM: bizarre and can't generate lots of common SQL you will want to generate. Weird joins via counting underscores. Horrible errors that don't make sense. Lots of builtin functions have never worked and they won't fix them. (ROUND has never worked!)
- Template language: Slow and borderline unusable. are 50 pages of recursive nonsense tracebacks that doesn't tell you anything usable.
- Web app (controller) framework: Terrible and inflexible
I suggest:
- ORM: Sqlalchemy: probably one of the best ORMs in any language, good performance and very flexible. It won't get in your way much.
- Templates: jinja2: basically identical to django templates but faster and usable tracebacks are perfect
- Web app: Flask or werkzeug, but there are lots of good options here. Django is the worst.I've been at this a while: https://news.ycombinator.com/item?id=1490415
I'm not sure how much it adds to the discussion but so have I. Although I've taken a step back from doing web dev directly in the last few years, I have been a Django dev since about 2006.
But before we get into that kind of silly comparison let me put it another way.
There are plenty of people more experience than both of us, that hold Django in high-regard. I know developers tend to hold strong opinions about their tools and that's all fine - but you are rather hyperbolic about several matters that seem to be closer to personal preferences than they are absolute truths.