Show HN: Godb is a simple Go ORM for PostgreSQL, MySQL, SQLite and SQL Server
github.com
github.com
In that it is similar to LINQ or HQL or SQLAlchemy Core — at an even lower level really since things like conditions are provided as parameterised strings.
The purpose is to be more productive than doing all this job manually.
Many things can be improved, it's sill a young project. This project is initially a learning one : doing stuff to improve my skill, on my spare time. I tried to build something clean, and used it on some real data. Perhaps it could be useful for somebody else. Nothing more ;)
That's a technicality, it just loads query data into objects which is what more or less every query builder does, the objects aren't involved in any of the logic. By the benchmark you use psycopg is an ORM (it really isn't, it isn't even a query builder).
Don't misunderstand, I'm not saying a query builder is bad, I'm just saying this is not an ORM, and shouldn't be sold as such: it's going to disappoint people looking for ORMs like Django's or Rails's and it's going to turn off people who object to ORMs[0] but would have been interested in a query builder.
Not to mention a good query builder is an excellent basis for a full-fledged ORM.
[0] and there are many people who think ORMs are a terrible idea
Not sure about terrible but it is not a bright idea to be object centric when schema remains king. Using ORMs makes perfect sense when your domain model is canonically represented by the object tier /and/ you are reusing these (common/core) domain objects to build a multiplicity of applications.
Query builders build queries (like SQLAlechemy's core library). ORMs know how to marshal object attributes to/from database columns. None of this is to say that one is better than the other.
Do you have any links perchance? Because I can't shake the feeling that I'm a pariah among the developer community for thinking this way.
In the functional world, there obviously aren't objects and structs are not the same things as objects in the OOP sense.
Even then the key part of a ORM is the "R". ORMs allow you to utilize a RDBMS transparently as if you were manipulating the OOP objects.
A couple of projects that this is closer to than Django ORM.
https://github.com/elixir-ecto/ecto -> "Database Wrapper" http://www.yesodweb.com/book/persistent -> "data store interface" https://hackage.haskell.org/package/groundhog -> "Database Mapping"
I appreciate this guy's work, but DSLs make more sense to me than orms in golang.
it's been in development for years, its used in production by many companies, and theres been a lot of work to keep it lean and powerful through lots of iteration.
Goa is just for generating apis. Not for databases?
Why this kind of project ? Because I used Active Record (Rails), Entity Framework, played with Django ORM, ... and regretted that nothing similar exist in Go world. Instead of complaining I choose to do something, not building a full ORM by myself, but at least build something allowing me to have a better comprehension of the constraints (except of the classic "no generic").
I didn't choose to build another Go library thinking I'll build something better than others. I learned, and I got a library ;)
One thing you can say about Go is it has 1000 Micro-frameworks, ORMs, Routers, and Logging libraries. Most of them are within a percent of giving the exact same feature set with the exact same approach and and ever so slightly different API. Lots of them share the same mistakes as they are a user's first project or a learning arena (though routing has been a pretty benchmarked area so that's been better than the others).
As someone who has open source libraries, and who uses Go in their day job and at home my personal opinion is that Go needs fewer, stronger libraries. Well supported, well maintained pieces rather than this illusion of choice that's presented today. I'm sorry if I came off as crass in my reply to the OP but that's simply the state I see in the community today.
Ideally there'd be 2-4 ORMs in Go that would be the goto. And half would use a code-generation approach, half would use the reflection based approach.
https://github.com/astaxie/beego/tree/master/orm https://github.com/cosiner/gomodel https://github.com/jinzhu/gorm https://github.com/go-gorp/gorp https://github.com/coocood/qbs https://github.com/go-reform/reform https://github.com/vattle/sqlboiler https://github.com/upper/db https://github.com/astaxie/beedb https://github.com/naoina/genmai https://github.com/monochromegane/argen https://github.com/square/squalor https://github.com/theduke/go-dukedb https://github.com/dropbox/godropbox/tree/master/database/sq... (...and more) And now: https://github.com/samonzeweb/godb
Do we truly need this? This one in particular feels a lot like upperdb. And if it's for learning only, don't publish it to things like hackers news which makes people seem like it's something you should use as opposed to the more mature libraries out there. This is what I meant by fragmentation, it's an illusion of choice. There are actually very few logging/routing/orm/micro-framework libraries you should really use if you're serious about a project.
I have a very high bar for open source personally and I tend to view it "more hardcore". I think that if you're putting open source out there to a wide audience such as this that you have a contract with the people you're putting it out to that you're going to maintain what's there and ensure it's of high quality. And if you're going to go to that level of effort why not contribute to something that's already mountains of work ahead of you and effectively the same code anyway (upperdb/gorp/gorm/xorm in this case)?
I have many Go repositories on my Github account that I've used for learning - but I've never put any of them out there to wide audiences that were solved by a more popular project in the same way - the approach has always been radically different. When I couldn't find logging libraries I liked I tried using others and contributed to them. I've also never put anything out there that I don't intend on maintaining basically forever. So my behavior is consistent with my viewpoint for whatever that's worth.
My colleague suggests that other people use open source for different reasons and view it much differently than me, and that's possibly the misunderstanding here. I shouldn't try to reflect my opinion in that sense on others and I do apologize for that. I suppose "if it were me" I wouldn't have done what he has, and I was commenting as if people knew and understood that as a commonality - which isn't probably true.
Also - I think you hugely overstated my bias for my own library. I'd be just as happy if he had not posted this and worked on gorp instead. /shrug.
Also I've been waiting for a long time for Go's libraries to be converged on - hasn't happened yet :)
Maybe something with two tables: an "authors" table and a "books" table. Then an example of getting authors each struct of which has a `[]Book` of books.
It's composition tool, not a relational one. Sorry.
Isn't the R in ORM for relational?
> Isn't the R in ORM for relational?
I spoke about the nested structs. You can have nested structs for something else than relational mapping.
As I replied to masklinn , I was not sure how to qualify godb, and I admit that ORM is not a perfect qualifier.
No, ORM means Object Relational Mapper. if your ORM doesn't support relations between tables, then it is not a ORM.
Given a relationship between 2 tables, an ORM will at the very least fetch related records when rows from a table are loaded.
> godb does not manage relationships like Active Record or Entity Framework,
Godb is therefore NOT an ORM. The title is misleading. People can't just make up definitions like that. It's either an ORM or it is not.
Godb looks like a simple query builder.
There's no relationship between 2 tables. You're probably referring to foreign keys, but these are just constraints, not relationships in the sense you're alluding in your comment.
Object Relational Mapper: It maps Objects to Relations.
Everything else is just auxiliary functionality.
Although I get what you're saying, and indeed the facility you describe is very convenient, it's not what the term means.
I'm just pointing that out because obviously people conflate relation(which comes from relational theory) with relationships.
This is really RDBMS 101 and we do have specific names and terms for a reason, please do not conflate them and don't confuse other people.
When you will go to https://en.wikipedia.org/wiki/Entity%E2%80%93relationship_mo..., you will see that it doesn't even mention ORMs.
Now go to https://en.wikipedia.org/wiki/Object-relational_mapping.
Read the "challenges" section that points to https://en.wikipedia.org/wiki/Object-relational_impedance_mi....
Now I'll just quote this:
<<The object-relational impedance mismatch is a set of conceptual and technical difficulties that are often encountered when a relational database management system (RDBMS) is being served by an application program (or multiple application programs) written in an object-oriented programming language or style, particularly because objects or class definitions must be mapped to database tables defined by relational schema.>>
Open any DBMS book. Pick one at random and read the definition of a "relation". It's always going to be "table".
Alternatively, go to the most(?) popular ORM framework in the world and read this page http://hibernate.org/orm/what-is-an-orm/
More: https://www.fullstackpython.com/object-relational-mappers-or...
<<An object-relational mapper (ORM) is a code library that automates the transfer of data stored in relational databases tables into objects that are more commonly used in application code.>>
I could go on for hours, you get the idea.
Yeah I already said that: https://news.ycombinator.com/item?id=13184341
Now godb is just "a Go query builder and struct mapper".
Thanks for all comments :)
Reflection is a real drag on performance.
With Go the cost is ridiculous compared to round trips with database server and query execution. More productivity for just a bunch of CPU cycles. Of course each use case and constraints are different, they're is no universal rules.
Entity Framework... that is slow ! I don't know for EF Core, but EF6 is really a slow thing.
(disclosure: I'm the author)
Reform has enough methods for the daily routine queries. And also it's easy to use raw SQL queries with reform if we need them.
I know these are meant to decouple you from the db implementation, but don't most projects of sufficient complexity end up getting tied to one db anyway?
For one, you may get massive, massive developer productivity by eliminating loads of queries. In a lot of ORMs you do things like myObject.save() instead of generating a insert/update query for the particular state of myObject without taking into consideration which fields of the object is currently set. This reason alone enough for me to use them.
Second, bunch of additional functionality usually come out of the box with ORMs, such as JSON or arbitrary (de)serialization of columns, caching, logging etc which you would have to spend time on custom implementations of these otherwise.
Third, like you mentioned, some allow very easy change of database vendor.
Fourth, mental abstraction from the underlying data storage (you stay in your language's domain) which again contributes to the productivity.
Some other benefits include not seeing large SQL strings in your code which make it more difficult to read and maintain, security and ability to easily reason about what the code intends to do.
I don't want to sound arrogant (not my intention at all) but I believe those who don't understand or downright rejects the benefits of a decent ORM system are either the ones that never used one to see the productivity benefit or those who work on projects that need to squeeze the last bit of optimization. In a large enough project it is seriously pain and time consuming writing all those queries which could be elegantly avoided.
That said, concatenating raw strings to form SQL that has no compile time type-checking is tedious and error prone. Thankfully, my "expert" knowledge of SQL does not blind me to the benefits of what a well-written ORM can do for productivity. (E.g. results caching, compile-time validation, auto-completion in the IDE, etc)
I know SQL and actually often enjoy using it. I guess this is a common trait for Java developers.
Still I often find a ORM to be a good tool on many projects.
The only substitute I'd accept is a quasiquoter that checked my SQL syntax for me at compile time.
Some of the additional query languages might be able to do it (they could certainly do chunks of what I said), but they'd still be pretty klunky about it since they bodge it on the side, and still don't compose anywhere near as well as they could or should.
Mind you, I'm not sure this library can do it either; SQL is also quite difficult to wrap around because of its structural deficiencies. Trying to hack away foundational issues at higher levels is always messy, error-prone, and still filled with the quirks that shine through.
Something like:
CREATE VIEW permitted_X AS
SELECT X.*
FROM X, user
WHERE X.user_id = user.user_id
AND (X.permission OR user.is_super);
Other abstraction mechanisms are user-defined functions.
[1] https://www.postgresql.org/docs/current/static/sql-createvie...
Some of these things are fixed up by the product-specific languages, but only some of these things.
Here's another example; using your product-specific language, can you create a table for me with a variable number and types of columns based on the parameters passed in? I don't know them all, but I bet it's hard in most or all of them. No credit if your product-specific language lets you bash a string together and then somehow execute it; I'm calling for everything to be done via first-class mechanisms. (Also, I'm not asking for whether this is a good idea; it is obviously a tricky thing of dubious use. But that should be a software engineering determination, not a language restriction.)
PREPARE stmt1 FROM 'SELECT X.* FROM ? X, user WHERE X.user_id = user.user_id AND (X.permission OR user.is_super)';
EXECUTE stmt1 USING 'yourtable';
It's also not enough to know SQL. You also need to know how your ORM maps objects to the database. E.g. do you get the same object if you execute a query twice? A surprising number of people don't know, and assume that ORM objects behave like normal objects and that is not always the case.
Personally, I prefer to write SQL in my models as it just _makes sense_ to me, but I understand why some don't.
If you start creating custom views and queries then you need to update them whenever there's a change to your objects. Also getting views into source control is something that you'll have to manage on your own.
An ORM is just less work. Although you'll often need to drop into real SQL for performance reasons or for complex work.
All the good ORMs allow this.
ORM does the reverse, it tries to map objects into relational data, the problem is that those are two incompatible way of storing data, you can do this for some objects but it doesn't apply for all cases.
Since relational algebra was proposed by Cobb, it took over all databases because it was proven that it's most efficient way to store data we know.
ORM might be useful for some cases, but it will generally limit you with what you can do with data.
IMO I think what's really missing is to have have another query language that is integrated with your programming language (so you don't have to write it as a string, and can benefit from language features such as auto completion, type checking etc).
I kind of wish QUEL[1] would win instead of SQL, since that language seems to be easier to be integrated.
There are also attempts such as LINQ[2] which appears to do good (I did not use it myself, since I don't program in .NET), but it looks great, I wish it became a standard and got integrated in other languages.
After you know the above, you can use that SQL knowledge in any language you want when accessing the db. No need to learn a new ORM per language or anything.
Also, at this point, you are likely to start understanding more about scaling systems which involve intelligently choosing indexes, replication, and sharding. Now you are on your way to Jr. DBA status :)
That was my path at least. There are likely better ways.
From there the distinction between ORMs and query builders gets fuzzy, it's more of a spectrum than a bright line. I particularly like SQLAlchemy's model of a core SQL-building library with an ORM you can use piecemeal around it.
You can build nested queries, doing some extra work, but it's not elegant and will not work for any cases (especially with PostgreSQL) : https://gist.github.com/samonzeweb/b1ae3e2d1f3203ee1722ac4bd...
Some work to do to add proper nested queries into godb, but for a so simple example it doesn't seem to be extremely hard.