mkleczek

What I am trying to say is that even if computers are going to be able to solve todays database design problems (which I am sceptical of by itself) - we are going to have bigger/more capable databases and other problems that computers are not able to solve.

It is going to happen by definition - people by definition want more than they have today and want to solve unsolved engineering problems ("problems" that machines can solve are simply not problems anymore).

The only way it can end is creation of some overwhelming intelligence (AGI) that would govern (or some might say: enslave) people so that their demands are controlled.

To be honest, now when I'm thinking about it - it seems to me Nix main weakness here is that it is a separate language and runtime.

Writing a Maven plugin (ie. a reusable piece of configuration management logic) is easy because you can use any library from the vast Java ecosystem (just add a library as a dependency of your plugin).

Doing the same in Nix requires recreating these libraries in Nix.

Looks like Maven might simply be the right choice...

I don’t believe it will happen at all. There will always be some problems that only a human can solve because it’s a human that defines them.

It is always _by definition_ a game between humans where computers are just tools.

At least as long as AGI does not happen…

Until a computer can solve the problem “how to redesign this particular database design (tables/partitioning/indexes/queries) so that it can handle expected load in next 5 years” I can safely assume it does not hold :)

That would mean 95% of problems are already solved.

I don’t agree with this - the space of problems to solve is as big as that of solutions (both are programs in the end).

I don’t think suggestion about lack of experience deserves any response.

I am quite confident my job is not in danger. Sure - scaffolding and boilerplate can be automated using ML - but problem solving - not at all.

The reason is simple - to find a solution is to create a program that has a specific set of properties. This - by Rice's Theorem - is not possible to realise using a Turing Machine. So as long as the solution search space is small (ie. we are talking about very similar programs to generate) - ML based methods are fine.

But once we are talking about finding solutions to non-trivial problems - I am sure human is needed.

Indeed - that’s very often the case - the right balance between too many and too big is not easy to find.

Having said that - Java is in general IDE targeted language and once you have an IDE - many small files is not an issue anymore.

There are attempts like

https://flake.parts/

or

https://github.com/nix-community/flakelight

Their aim is to create an ecosystem of reusable Nix libraries. But it is tiny.

In Java/Maven ecosystem a lot of things is simple because there is a huge ecosystem of easy to integrate libraries/plugins.

Want a packaged spring boot application? There is a plugin for that. Want to package it in a container image? Just add a plugin dependency. Want to build an RPM or deb package? Add another two plugin dependencies. All various artifacts are going to be uploaded to a repository and made available as dependencies.

Missing a specific plugin? You can easily implement it in Java as a module and use it inside your project (and expose it as a standalone artifact as well).

I can’t find anything similar in Nix ecosystem.

Having a language allowing for this is not the same as having solutions already available.

Nothing got lost: both Nix and Maven are dependency management tools. Both are also build tools. The difference is that Maven was created as a Java build tool (and it stayed that way in general).

What we have today is that there is a multitude of dependency managers and - what's worse - all of them are _also_ build tools targeted at specific language.

Nix has a unique position because it is not language specific. Where it is lacking is missing standards and reusable libraries that would simplify common tasks.

I am comparing to Maven because I am looking for multi-platform Maven alternative. There is Bazel but its dependency management is non-existent. There is Buck2 which is great in theory but the lack of ecosystem makes it a non-starter.

Nix is the only contender in the space that offers almost everything and has a chance to become a de-facto standard of software delivery thanks to this.

What's missing though is... easy to use canned solutions similar to maven plugins.

EDIT: grammar

The question is if there actually _is_ a rich ecosystem of such libraries - similar to the rich ecosystem of Maven plugins.

And what's the story with generic libraries that can later be used in your nix files to produce desired output?

I am aware of flake-parts that are supposed to offer that but the ecosystem of flake-parts is on the smaller side of things...

Does Nix make it difficult to properly modularise nix files (flakes or not)?

Because it certainly looks like the things you listed are separate/orthogonal and should be in separate modules/files.

Having many years of Java experience this is the reason why I stick to Maven (not moving to Gradle) - it is opinionated and strongly encourages fine grained modularisation.

> Often in the real world you need tree-like structures, which are fundamentally not able to be represented by a table/relation. No amount of joining can produce anything other than a table/relation. The n+1 problem is introduced when you try to build those types of structures from tables/relations.

You can easily get hierarchical output format from Postgres with its JSON or XML aggregate functions.

You can have almost all benefits of an embedded database by embedding your application in the database.

Just change perspective and stop treating Postgres (or any other advanced RDBMS) as a dumb data store — start using it as a computing platform instead.

To say it is light on details is to say nothing…

But big claims require much more and the claims are huge (ie. It is a perfect DBMS)

I’ve taken a brief look at the website and the GitHub repo. Could not find any description of the architecture, algorithms used, any references to research papers or anything like that.

It does not build any confidence as the topic of distributed highly available and scalable databases is inherently difficult and solutions are full of trade offs.

IMHO the only really interesting alternative to Git currently is Pijul (https://pijul.org) as it is not a more-or-less Git clone but a different approach to the problem itself.

Pijul allows for very interesting development and ci/cd workflows.

> Apart from that, it looks like Java RMI allowed remote objects returning other remote objects, rather than only immutable values.

Actually this is a very powerful concept as it allows one to achieve high level of reuse. Jini (https://jan.newmarch.name/java/jini/tutorial/Jini.html) made mobile objects the core idea of its architecture.

How powerful the approach is one can see when looking for example at Jini concept of a Lease and LeaseRevenewalService:

a server program can register an object (client side implementation of a service) in ServiceRegistrar (to make it discoverable and downloadable). Registration is lease based so the server has to renew it periodically. But it can be delegated to a LeaseRenewalService that can do it on its behalf so that the server can go to sleep (ie. not use any server machine resources).

All of the above happens without any party a-priori knowledge about any code that needs to be present at use site - code is downloaded automatically on-demand - the only thing common to client and service is a Java interface.

Let's be honest - deployment story with J2EE application servers was total mess and I still have nightmares.

But you're right - today's world of YAML programming is no better.

Nix and Guix are steps in the right direction (but both have their fair share of issues)

Mobile code and "agent systems" were very fashionable 20 years ago. Java introduced built-in RMI with automatic stub downloading. In 1998 Sun published https://en.wikipedia.org/wiki/Jini that was an extension of this idea. Several higher level frameworks emerged (JavaSpaces among the most prolific).

Reading the paper two thoughts come to mind:

- "What's old is new again"

- "those who do not learn from history are doomed to repeat it"

There were several attempts in history to implement transparent RPC (new again). All failed due to https://en.wikipedia.org/wiki/Fallacies_of_distributed_compu... (learn from history)

Looks like any abstraction trying to hide distribution is inherently too leaky.

I think it would be worth mentioning GraalVM and https://github.com/oracle/truffleruby in competitors section.

What's old is new again but in new clothes.

20 something years ago "mobile objects" and "agent systems" were a thing and one of the practical incarnations was Jini: https://jan.newmarch.name/java/jini/tutorial/Jini.html

> if you want to design classes for serialization that is agnostic to serialized format

You mean XML/JSON/binary etc? This is impossible :) - the only thing that comes close is ASN.1.

But serialisation can be easy: just don't do anything and let runtime handle that. Any compacting GC can be seen as serialiser/deserialiser (it does move object graph from one place to another) - and it does not need to run any application code to perform its task. Criu and any other live program ("object") migration methods are similar.

> and that accounts for cases where not both sides of the communication are trusted

I don't believe Java serialisation will ever be capable of handling deserialisation of untrusted input.

The premise of the sentence you cited is that it is necessary to call application code (constructors) to maintain security/integrity.

What I am saying is that it is NOT necessary to implement secure serialisation and deserialisation because you CAN (and should?) implement it at a different layer and transparently to the application code.

The point is that security and bypassing constructors are completely orthogonal as serialisation by memcpy is completely viable strategy and is no less secure than calling constructors on objects.

This is really exciting and I hope to try it out at my company ASAP.

Why?

Zero copy state transfer is a viable and high performance alternative.

Security and integrity can (should?) be implemented at a different layer.

If by "an API" you mean "JSON sent over the network" then I don't really see too much of an advantage in: database -> server side program -> JSON -> network -> client side program -> HTML -> browser renderer

over

database -> server side program -> HTML -> network -> browser renderer

Well, it depends on the language. Some are quite good at restricting programs so that it is not possible to execute arbitrary code.

Take a look at https://propellor.branchable.com to see how Haskell might be used.

Idris might be a good candidate as well.

https://dhall-lang.org is quite interesting for these purposes as well (although it is not general purpose)

While deceptive at first... Just try to imagine how "extract method" refactoring would work...