1,882 karma · joined August 30, 2009
I use Arch, btw ;)
I have a friend who maintains a gazillion JVM based apps for the Norwegian tax authorities and he's a Kotlin skeptic. He's pretty tired of maintaining and porting various small apps and microservices from arcane old and discarded Scala web frameworks etc. Given how locked-in Kotlin is to a single company, I kind of get his skepticism.
I just got a dedicated server on Hetzner to test out some things. It's $70/month with 64gb RAM and a CPU that builds a complex C++ thingie in 11 minutes where my laptop spends about an hour :D
Scaling is of course not trivial, but the same set of backoffice apps I've worked with throughout the years that would be a good fit for Datomic, has a working set for the database much smaller than 64GB.
> they seem like niche features
That's the thing, though. Maybe it's because I've use Datomic a bunch. When I'm on projects that use a SQL db, a handful of problems are just fundamentally solved in Datomic, and none of the super knowledgeable devs that know SQL in and out are even aware that they are problems.
Some examples:
- What caused this column of this row to end up with this value?
- Oh no, we were down 3am but when the first person investigated it at 7am (oh those backoffice SLAs...) everything works fine and nobody knows that the db state was at 3am
- When we wrote value X, what other values did we also write at the same time?
- We need to carefully write a transaction that reads some data, writes some data, and reads some more data, and hopefully lock correctly, and hopefully we understood the isolation level (that we probably didn't set anyway) correctly and...
Which makes me think I must be the crazy one...
Interesting framing. The people behind Clojure and Datomic aren't known for being amazing at scaling and shipping products (i.e. marketing and all that jazz).
This is also not me judging them for it, I haven't built and shipped a Datomic, much less marketed it.
I presented on Datomic at KotlinConf too, with some live coding starting around the 31 minute mark https://www.youtube.com/watch?v=hicQvxdKvnc
You probably need to be on the JVM, as the peer library (i.e. the "good one", where you embed the full query engine and data fetching directly into your business logic) is so far only implemented for the JVM.
I suppose 10+ years of weird license models and a hefty price tag haven't helped. Datomic turned free (but still proprietary) in 2023 though. But why Datomic isn't more widely adopted is a huge mystery to me...
What part needs a cluster in your experience? The Datomic deployments I've been involved with have been running on a single server, with a single instance of backing storage etc.
Datalog is interesting indeed... Back when I first started using Datomic in 2012 I had just fundamentally decided to use it, and it took probably a week before the query language "clicked", i.e. to be able to actually compose my own queries and not just copy paste my way to something that works.
I get that postgres is a good default in many cases, and I don't expect SQL to die tomorrow. But there are _so many_ apps (most/all backoffice apps I've worked on for example) with 10s or 100s of transactions per second at most, that would love to have the data available directly inside your business logic, and where both business logic and devops would improve by many orders of magnitude by having a full transaction log of all changes done to your data over time.
Is it _just_ because Datomic is different and people don't get it, and that preconceived notions makes you think Datomic is something it isn't?
Here's to the crazy ones!
Which bridge that is currently in operation should be closed next?
(Not a dunk on the article, which brilliantly addresses the difficulty of knowing in advance vs making real world changes. Practical Engineering is an awesome YouTube channel!)
Has this improved since 11th gen Intel?
Datomic's covering indexes are heavily based on their built-in ordering, and doesn't really have much flexibility in different ways to sort and walk data.
Personally, I'm a fan of hybrid database approaches. In the world of serverless, I really enjoy the combo of DynamoDB and Elasticsearch, for example, where Dynamo handles everything that's performance critical, and Elasticsearch handles everything where dynamic queries (and ordering, and aggregation, and ....) is required. I've never done this with Datomic, but I'd imagine mirroring the "current" value of entities without historical data is relatively easy to set up.
Additionally, this laziness means your business logic can dynamically choose which data to pull in based on results of queries, and you end up with running fewer queries as you'll never need to pull in extra data in one huge query in case business logic needs it.
However, if you want to paginate data that you need to sort first, and the data isn't sorted the way you want in the index, you have to read all of the data first, and then sort it. But this is also what a database server would need to do :)