The guy like you on a mission critical team at a cutting edge company is a godsend and will be a big part of why the project/company succeeds. The guy who wants to build his own ORM for his no-name company's CRUD app is wasting everyone's time.
The guy like you on a mission critical team at a cutting edge company is a godsend and will be a big part of why the project/company succeeds. The guy who wants to build his own ORM for his no-name company's CRUD app is wasting everyone's time.
I once unfortunately joined a project where an off-the-shelf ORM had been selected, but when development was well into the deep edge cases started to reveal serious design flaws in the ORM library. A guy wanting (perhaps not a in a joyful sense, but more not seeing any other choice) to build his own ORM that was mostly API-compatible was what saved the project.
This was a long time ago. The state of ORM libraries is probably a lot better today. But the advice of ensuring that a library is SQLite-grade before committing to it does rings true even for simple CRUD ORMs. Perhaps especially so.
One of my favorite features of Entity Framework from my .NET days is that it's very easy to just break out of the ORM functionality, even from within an EF-specific function, or to have multiple instances with slightly different configuration (I never had to do that last bit but I know it was possible a decade ago).
But in your example, even if an ORM doesn't provide native breakout functionality, it should be obvious that you can maintain a bespoke path to the database for cases where the ORM doesn't fit. Where that isn't obvious to someone, perhaps 'creating their own ORM' isn't the waste of time you make it out to be, but is actually necessary education?
If you are No-name CRUD Company you're probably not hiring the million dollar per year devs who have learned all the lessons. You kind of have to accept that you are working with budget workers and thus learning on the job is going to need to take place, even if you wish that weren't the reality you face.
At the risk of going off on a tangent, the median dev salary is something like $100-150k/yr. So half of devs in the country make less than that. Gergely Orosz has a great discussion of this.[0] $1m/yr TC is the top 0.01% of the top tier of companies. Some FAANG-level tech firms are here but otherwise it's almost entirely IB, HFT, hedge funds, that sort of thing. I'd be shocked if anyone making close to $1m/yr TC is ever touching an ORM in their day job.
[0] https://newsletter.pragmaticengineer.com/p/trimodal-nature-o...
But, as I said, even if it isn't built-in, it doesn't make any difference, does it? Either way there is no reason to throw a perfectly good ORM out the window just because in some cases it isn't the right tool for the job. Surely you agree? That was my interpretation of the intent of your earlier comment.
While it may be true that ORMs today are of SQLite quality, the original commenter's point still stands: You need to make sure that is the case, else you are going to quickly wish that you did write it yourself.
> So half of devs in the country make less than that.
You may take things a bit too literally, but if you want to go down this road, do you truly believe that half of all devs have learned all the lessons there are to learn in software? Color me skeptical. 0.01% is likely even pushing it.
Even back when it was launched EF was miles ahead of most mature ORMs of today, and I believe your 95% number. But other than EF plus a handful of other mature ORMs, the 95% number looks more like 50%.
I would even argue that new-ish ORMs are virtually useless for anything that's not CRUD, and that the CRUD part can be 100% replaced seamlessly by something like PostgREST/Supabase or Hasura without losing much.
I don't disagree with the feeling in general, but I feel like we are making mistakes by having as much faith in modern ORMs and even libraries in general. Veeeeeeery few things even come close to being 1% as good as Entity Framework, ASP.NET, Rails, Postgres or SQLite.
I have a side project that uses Clerk for auth but basically every other supabase product there is and it really is great for smaller use cases. I don't know how it stacks up once you start needing really fine-tuned database permissions or functionality though.
I find it that if you accept Supabase as-is, it can get you pretty far and save a lot of time and money.
And for edge cases, it's like you said above about ORMs, we don't have to throw it out, we just handle those cases separately.
Slowly replaced it with Dapper and handwritten SQL, a simple migration versioning system, and database seeding with validation. Once that was done, startup time was cut by more than 10 seconds on a standard SSD and about 30 on CFast. Even finally replacing the database connection with SQLite standard libraries shaved off 2 seconds.
EntityFramework maybe useful but it lacks performance when time to start using the software is important.
There are a large number of fundamental impedance mismatches between relational data and object based data. Any ORM can fix some of them at the cost of ignoring others, but the fundamental character of ORMs is such that taking an opinionated line on tough tradeoffs is as good as you can hope for.
This is why ORM guy is wasting everyone's time - his problem is almost definitely not going to have a unique or even valuable perspective on all of those tradeoffs.
Should your application map objects and relations at all isn't usually a question you get to ask unless it is doesn't do much or lives on its own private island. Should you do it yourself or lean on a toolkit to help is the question that you have to contend with.
Which is no doubt why most newer applications I see these days have trended towards carrying relations as far as they can go, only mapping with objects at the points where it is absolutely necessary.
> It's also rarely practical to take on the project of making a slightly different set of compromise
I suppose that is the other benefit of delaying mapping until necessary. What needs to be mapped will be more limited in scope and can be identified as such. You don't have to build a huge framework that can handle all conceivable cases. You can reduce it to only what you need, which is usually not going to be much, and can determine what tradeoffs best suit in that. In this type of situation it is likely that using a ORM library is going to be a bigger waste of time, honestly.
I believe Active Record is a more specific implementation of something that is ORM-like. We can stop speaking of Active Record since my point holds for the more generic ORM, and therefore holds for Active Record as well.
To clarify my point, there is a fundamental impedance mismatch between object mapping of data vs relational database mapping of data. One implication of this is you cannot use database as a service. Interactions with database must instead be gated behind the ORM and the ORM controls the database interaction.
I'll note that database as a service is very powerful. For example, when there is an API contract exposing a value that is powered by some raw-dog SQL, when the database changes, anything using the API does not need to change. Only the SQL changes. In contrast, when an ORM exposes an object, an attribute might sometimes be loaded, sometimes not. A change to load or not load that attribute ripples through everything that uses that object. That type of change in ORM-land is the stuff of either N+1 problems, or Null-Pointers.
To back up a bit, let me re-iterate a bit about the impedance mismatch. Wikipedia speaks of this [1]: "By contrast, relational databases, such as SQL, group scalars into tuples, which are then enumerated in tables. Tuples and objects have some general similarity... They have many differences, though"
To drive the point home - in other words, you can't do everything in object world that you can do in a database 1:1. A consequence of this is that the ORM requires the application to view the database as a persistence store (AKA: data-store, AKA: object store, AKA: persistence layer). The ORM controls the interaction with database, you can't just use database as a data service.
I believe this point is illustrated most easily from queries.
To illustrate, let's pull some query code [3] from Java's Hibernate, a prototypical ORM.
```
public Movie getMovie(Long movieId) {
EntityManager em = getEntityManager();
Movie movie = em.find(Movie.class, new Long(movieId));
em.detach(movie);
return movie;
}```
So, getting a release year might look like this:
```
int movieId = 123;
Movie m = orm.getMovie(movieId);
return m.getReleaseYear();
```
In contrast, if we put some raw-dogged SQL behind a method, we get this code:
```
int movieId = 123;
return movieDao.getMovieReleaseYearByMovieId(movieId);
```
Now, let's illustrate. To do this, let us look at the example of finding the release year of the highest grossing movie. As a service, that looks like this:
```
return dao.findReleaseYearOfHighestGrossingMovie();
```
In contrast, as an ORM, you might have to load all Movies and then iterate. Maybe the ORM might have some magic sugar to get a 'min/max' value though. We can go on though, let's say we want to get the directors of the top 10 grossing movies. An ORM will almost certainly require you to load all movies and then iterate, or start creating some objects specifically to represent that data. In all cases, an ORM presents the contract is an an object rather than as an API call (AKA, a service).
For the update case, ORMs often do pretty well. ORMs can get into trouble with the impedance mismatch when doing things like trying to update joined entities. For example, "update all actors in movie X". Further, ORM (and objects) creates issues of stale/warm caches, nullity, mutability, performance, and more... What is worse, all of this is intrinsic, relational data and objects are intrinsically different.
[1] https://en.wikipedia.org/wiki/Object%E2%80%93relational_mapp...
ORM and entity manager – which, in turn, is a query builder combined with a few other features. Your code is really focused on the latter. While the entity manager approach is not the same as active record, that is true, the bounds between query building and ORM, I think, are even clearer. In fact, your code makes that separation quite explicit. I can at least understand how ORM and query building get confused under active record.
> We can stop speaking of Active Record
While I agree in theory, since we are talking about ORM only, if we go by Wikipedia we cannot as is ends up confusing active record and ORM as being one and the same. That is a mistake. But as my teachers, and presumably yours too, told me in school: Don't trust everything you read on Wikipedia.
But we don't need to go to Wikipedia here anyway. Refreshingly, ORM literally tells what it is right in its name. All you need to do is spell it out: Object-Relation Mapping.
My point is that (bluntly speaking), ORMs are intrinsically fucked because relational mapping and object mapping are just fundamentally different. Because of that difference, some things will always be difficult when doing so in any orm.
Here is an example of something that an ORM does well:
```
Person p = entityManger.findById(123);
p.setAge(23);
entityManager.persist(p);
```
I suppose an active record example is something like:
```
Person p = Person.findById(123);
p.setAge(23);
p.persist();
```
Regardless, of Active Record or ORM, the above is doing this query:
```
update person set age = 23 where id = 123;
```
The above is simple. When trying to update a linked entity is an example where ORMs are going to have complexity. Let's say a person owns books, and books are unique in the system. This type of query:
```
update book set person_id = (select id from person where name = 'Joe') where person_id = (select id from person where name = 'Jill')
```
In code, that looks like this:
```
Person joe = findByName("Joe");
Person jill = findByName("Jill");
jill.getBooks().stream().forEach(book -> { book.setPerson(joe); book.persist()) });
```
The ORM code is so convoluted... We do a full select for two Person entities, possibly eager fetching all their books with more queries (N+1) problem, but all other entities attached to a person as well, and then we do 'N' update statements. These types of problems are AFAIK unavoidable. They will happen for one scenario or another. One can choose the object representation to mitigate one case or another, but it's not long before something that is trivial in SQL becomes a huge burden in OO.
Thus, my thesis, Object-Relation Mapping will create unavoidable cases of convoluted code because RDBMS do not have a perfect mapping to Objects. The mapping is not perfect,the result of this is intrinsic complexity that is easily solved by sticking to SQL, but very difficult in ORMs (which manifest as various issues of inappropriate eager vs inappropriate late fetching, N+1 queries, caching issues, transaction issues, etc..)
My experience tells me that the largest among these impedance mismatches is the inability for OOP languages to express circular dependencies without resorting to messy hackarounds. Developers often fail to realize how far they are into the dragon's den until they need to start serializing their object graphs.
It’s overkill for small projects and not expressive enough if you’re doing really complicated stuff. Even if you do have a good use case for an ORM currently as your requirements grow it gets harder to hack stuff on that you need.
That's a bit orthogonal. Even if you use an ORM library, you'd be remiss to not put it behind a DAL. But from your DAL if you emit/accept objects of your own transformation: Congratulations, you've just invented an ORM. You can emit/accept relations, which is quite justifiable, but even then you are bound to have to map it to objects at some point. e.g. interfacing with third-parties that require objects. There is really no escaping ORM in any reasonably complex real-world application.
Generally, you can go a long way with simple, combined types which are closer to maps/hashes/dicts than to "objects" other than syntax (.attr vs ["attr"]).
And really, that would be my preference: combine a query builder (some ORMs have great ones too) with native types representing the data read from the database.
Agreed. Of course, strictly speaking, a relation is specifically a set of tuples. But if you are working with a SQL database, which has been implied, you are already long past that idea, so it is understood that we're speaking of the concept somewhat more loosely. An instance of a class with a set of basic properties nestled in an array would still reasonably be considered a relation as it pertains to this discussion, as far as I am concerned, and seemingly you too. Fair to say you haven't meaningfully changed the semantics of the data in that.
But that doesn't mean you won't need to map to objects. You almost certainly will at some point in a reasonably complex application, even if only to interface with third-parties.
But as the idea and project cements itself, you start to see exactly where the biggest flaws are, and you might draw the conclusion that a lot of problems could be fixed at the ORM layer, so you opt for working on that.
Maybe it would have been obvious from the beginning, but chances are the people working on the codebase initially had a very different idea of what exactly is the ideal design, compared to later on in the lifetime of the project.
I would say it's not. Sure old ORMs still have their features, but newer ORMs and especially ORMs in newer languages have a fraction of the features of something like ActiveRecord or Entity Framework.
We respond to incentives. If a developer's only incentive is "we reward shipping as fast as possible" then they will carelessly ship slop as fast as they can type it. If that incentive is removed, they can learn a better way...
A lot of it is pretty basic: checking every single return code, testing every single branch, verifying that the external environment is doing what it claims or should be doing. All of this is muscle memory now, I find it difficult to write a throwaway python script without doing this. I also don’t feel like the degree of thoroughness I put into it significantly slows down development either compared to other developers that YOLO it a bit more; I spend a bit more time writing it, they spend a bit more time debugging it in test. And in prod, the former approach has lower defect rates.
It doesn’t need to be safety critical embedded software, which has a somewhat niche set of problems. Even fairly high-level data infrastructure has many of these same challenges.