Hacker News Reader

If you must use Django, use it through an abstraction layer like this:

https://adsharma.github.io/django-fquery/

Your models can be plain old python data classes, declaratively mapped to Django primitives.

I despise django-orm, doctrine is so much better. Like, who thought that using named arguments to do stuff was a proper way ??? `.filter(created_at__gte=XXXXX)` why? The rest of the framework is great but the ORM is definetly its weakest point.

Having used Doctrine, hard disagree. Never again.

I've used django-orm at my previous job for 2 years and I never have liked it, the syntax is just not nice to read. I used Doctrine for 5 years (2 before last job and 3 since I got my current one) and it's just night and day. Declaring entity is just way cleaner, you can just skim through an EntityRepository / query and understand easily what it does

I find those double underscore kwargs weird too, and would prefer to simply pass a lambda instead. What is your idea, what would you suggest?

The double underscore kwargs are a bit odd, I agree, but once you know about them they're okay.

And, rather like the petrol engine, it turns out while it sucks, everything else is massively worse in some vitally important way.

In Doctrine the query builder can take objects that describe what you want to do [1], not the best but still way better to read and understand. There's also the DQL which is an SQL-like language that's pretty well integrated in phpstorm and is quite close to SQL [2]

[1] https://www.doctrine-project.org/projects/doctrine-collectio... (for collections but you can use them for queries too)

[2] https://www.doctrine-project.org/projects/doctrine-orm/en/3.... / https://www.doctrine-project.org/projects/doctrine-orm/en/3....

I seriously don't get why Django ORM is using the Active Record pattern. This is such a stupid footgun that trivially causes horrible performance and BEGS you to cause n+1 problems.

Never in my life did I have a problem with lazy loading causing unbearable performance until I joined a Python Django team. I really tried to find sympathy for the "dynamically typed" folks (please spare me saying Python is technically statically typed), but coming from writing apps and backends in Java, Swift, C#, Objective-C and PHP, Python with Django was the worst experience bar none.

I worked on the project for 10 months, could at least refactor the project to something semi-sane where obvious mistakes (which would not be possible in other languages) could not happen. Then along comes a "good Python dev" and threw it all out of the window and start doing SQL queries all over the place (typically 3-6 lines long), remove the domain objects and cause the same problems I started with to begin with. But his approach was saying that the other developers were "not good enough".

Yeah, have fun with schema changes going forward. Good riddance.

> BEGS you to cause n+1 problems

select_related, prefetch_related. n+1 problems be gone.

You misunderstand. And that is exactly the problem.

We did do that and that's why our queries ended up being several lines long. But if you missed just one model? You openly walk a knife again.

It's a mess and it only gets longer and longer. I ended the project with having some proper aggregates, only for that to be thrown out of the window by the guy after me.

> our queries ended up being several lines long

Which is... perfectly normal for non-trivial needs.

> I ended the project with having some proper aggregates, only for that to be thrown out of the window by the guy after me.

How is that a Django problem though ? Sounds like a skill issue on your successor.

I get what you say, there's plenty of debates about ActiveRecord vs. AnythingElse, but in the end this one has its use and obviously has served many of us just fine. Different strokes... you know the drill.

I think there's an argument for throwing AttributeError instead of silently going to N+1 behaviour.

See sibling comment about fetch modes in coming 6.1 (https://docs.djangoproject.com/en/dev/topics/db/fetch-modes/), you can have a FieldFetchBlocked.

If you want to just prefetch everything throughout a project or just on a per-Queryset basis, all that is coming in the next release of Django.

https://docs.djangoproject.com/en/dev/releases/6.1/#model-fi...

That's the great thing about Django, it's been around so long and the quality bar is so high that eventually all the major rough edges get sanded away usually in a really well considered manner.

It still selects all fields by default. Very often I have to use defer() or only() to get rid of expensive columns like blob/text that are rarely used and greatly hurt performance when grabbing them.

Then it got to where I had to make a reflective function that I use like Model.objects.defer(*all_fields_except(Model, ['field1', 'field2'])), and then add another all_fields_except() for every select_related and prefetch_related.

Even save() by default re-writes every single field. You have to use save(update_fields=['field1']) instead.

Yes, if you attempt to use Django like you'd use your typical Java, Swift, C#, or Objective-C framework, you're not going to have a good time.

I've seen the horrors Java devs start doing on a Python project when trying to "fix" things, where by "fix" they mean use patterns they had to in previous gigs.

It's a different world.

Having basic defensive programming, some simple classes instead of dicts everywhere and avoiding n+1 is a "different world"?

Just asking these questions underscores lack of understanding how things are usually done in Django and Python in general.

In Django you'd typically use simple classes (models or forms, or even dataclasses nowadays) more than dicts everywhere; n+1 is trivially avoidable (as another sibling comment points out, and you also have multiple packages that autodetect such cases if you've missed them).

Python in general has a more "consenting adults" than "defensive programming" attitude (which doesn't mean exessive coupling or spaghetti, but the approach is different from the Java or C# mindset).

There's no one THE correct style of programming.

> Just asking these questions underscores lack of understanding how things are usually done in Django and Python in general.

No, it doesn't. It's fair to criticize the consequences of this approach to coding.

What would you have used instead?

The ORM is really not good in my opinion because it is ActiveRecord'ish and has all its downsides. I wouldn't use Django for any moderately complex domain. But even with simpler CRUD style apps I don't really see the point in it.

What would you have done Instagram from instead of Django?

Elixir and Phoenix.

You'd have written Instagram which was released in 2010 in Elixir which wasn't released to the public til 2012?

So Rails or some PHP framework. It was slightly too early to go full Node. Django was a little unusual too among the developers I knew. Java was still a thing but more for finance related projects.

Well 2010 PHP and the frameworks at the time were still going through the 5.x desert journey, and the prospects weren't entirely clear with the cancellation of PHP 6, so you wouldn't fault your 2010 self for not trying to push some Drupal/Joomla/Magento to that scale.

Kinda took until Facebook showing off Hacklang in 2014 for people to believe in getting more canonical programming features into PHP and make it more performant. So it would have been a good decision if one could predict 10 years into the future, but nobody can.

Rails was fully into growing pains and maintainability crises (some large rails codebases took years to migrate) and PHP was in transition; some good things by then but it was not what it is now.

All of the gripes OP has with Django are arguably worse in Rails.

I think Threads could have been done in Elixir.

Why would you have chosen these? What are the advantages?

The one thing I really appreciate with the ORM is that you really can get the ORM to make... more or less any sort of SQL query you want.

It can take a while to wrap your head around what fields get used in aggregates and the like, but when working with big models with like 65 fields and juggling a bunch of stuff, not having to futz with serialization/deserialization and "just" expressing your problem in the dumb way is nice.

I want to say this all comes back to bite you in the end but honestly it's more just having wide tables that comes to bite you. A service layer wouldn't really save you. Meanwhile you save yourself a bunch of tedium in the mean time

> The one thing I really appreciate with the ORM is that you really can get the ORM to make... more or less any sort of SQL query you want.

And if you can't make the ORM make the SQL query you want, you can just write it as a SQL query, like this godawful monstrosity:

      x = Site.objects.raw("select id, name, lat, lon, 111.045*degrees(acos(cos( \
        radians(latpoint))*cos(radians(lat)) \
        *cos(radians(lngpoint)-radians(lon)) \
        +sin(radians(latpoint))*sin(radians(lat)))) \
        as distance from sites_site join \
        (select %s as latpoint, %s as lngpoint) as p on 1=1 \
        order by distance limit 5", [float(lat), float(lon)])

... which calculates the Haversine distance from where you are now to the five nearest points.

I am in roughly equal parts proud of and horrified by this creation.

I'm not seeing anything that can't be done here without using raw() though?

Yeah I'm not clever enough to do that.

How would you have approached it?

It looks like GeoJSON implements some distance lookups that make this more django-y, that's probably the easiest way. But I doubt they're using raw() either.

Site.objects.annotate( distance=Degrees(ACos(Cos(.....))), latpoint=float(lat), lngpoint=float(lon), ).order_by("distance")[:5]

for function calls, look at django.db.models.functions, you can find a bunch of stuff in there or create custom ones super easily (like "two lines of codes" easily)

I mean you have a thing that works in theory so it's a bit of navel gazing, though.

That is actually awesome :-) I'll try that.

I was really just going to try and wrap it in a function to stick in the model so I can say something like Site.objects.get(id=thing).distance_from(lat,long) in.

But, now the bit that I was calling that from is actually being done from a websocket handler, and that's written in Go because Django and websockets seems very complex.

I mean if you're doing it this way, you're really not applying best practices as a developer (never mind as a Django developer).

> Models being passed around everywhere, queries happening everywhere.

No, as a developer you still need to be 100% aware of the underlying queries and potential performance issues. No excuse for N+1 problems. ORM is not an excuse to be lazy, but I admit it will probably catch quite a few developers.

Those same developers would probably make a mess out of any other framework or technology though.

Django allowing queries to be anywhere is more or less in line with Python’s overarching “we’re all consenting adults here” ethos. There’s probably one correct way to do it, but if you want to shoot yourself in the foot then here’s your gun.

It definitely takes a bit of discipline. The key layers are somewhat easy to manage—middleware, context processors, views, template tags—but I’ve seen some hairy lasagne further obscuring where the queries happen on top of that. A well-documented abstraction can be useful, but if it is possible to keep it simple and obvious then that’s the way to go.

(Third-party dependencies can further complicate things, but at least you can expect a library using ORM to be in the installed apps list.)

I'm semi-assuming we're talking about professionals who'd excel with their products in any framework and language.

It's highly productive if you do it right.

The other thing that I think people tend to forget is that there are plenty of situations where N+1 queries just aren't that big a deal. Not every view in every application that every developer builds needs to handle massive amounts of traffic with low latency and high cardinality tables. I've built so many apps where there's one or two users, small amounts of data, etc. And even on apps that do have a lot of traffic, there are often internal/admin/maintenance views that don't have the same requirements and no one will notice an N+1 where N = 5 in the worst possible case.

Every time ORMs get discussed, it seems to be dominated by people who are like "but my app has 5 billion concurrent users doing 2 million requests per second and if there's an extra 5ms on my requests, it will all explode!" and can't comprehend that not everyone is building the same kind of systems all the time. Great, maybe an ORM isn't appropriate for your situation.