If global, mutable state is considered problematic when programming in the small (i.e. single program), why is it unquestioned when building much larger, more complicated systems that we also need to reason about?
That's why the design discussions about how to do stuff on such networks always centre on how to manage this global, distributed, mutable state. It really is hard stuff.
The nice thing, is that once you have done that design work, and are programming "in the small" to implement it, the individual processes can be stateless.
The same goes to your filesystem, the key thing is: global as global access vs. global as "there is one of it". The same goes for the singleton pattern (which are terrible) vs. just having one of something.
Once you mutate the database, then the behaviour of any code anywhere that uses the DB, whether directly or through a module is potentially affected by the mutation.
Modularisation is great and all, but it doesn't make concurrency and cache-coherency problems go away. In fact it sometimes makes those things worse (or at least more opaque).
The internet is "global state" but you wouldn't call any code that makes HTTP requests global state because the "globality" is isolated inside your app.
Global can mean several things, my point was only that "globality" inside your app can (and should) be very constrainted while acknowledging some things are indeed global from your app's point of view. A good example of this is having a global "db" object vs. a DB instance passed to places.
This is more about whether or not your code should care about where it gets data or not. I think this (now classic) talk is a nice explanation about why your components should not "look for things" in the global but instead should be explicit about dependencies any why it's easier to reason about: https://www.youtube.com/watch?v=RlfLCWKxHJ0
Furthermore, if you deploy incorrect code, it could add JSON blobs to the database with the new schema before it crashes on the old schema. Now you've got two incompatible versions of the JSON blob in the DB at the same time, so rolling back to the old code will still be broken!
Software engineers deploying production apps have been dealing with these issues since we started using databases, but that doesn't mean that it's any easier or less error prone. It just means that we have to be more careful. And that care has a cost.
This cost isn't unquestioned for larger systems. It's a big part of the motivation for append-only databases, for example.
http://thesoftwarelife.blogspot.com/2010/12/append-only-data...
See for example Rich Hickey's Datomic [0].
[0] https://www.infoq.com/presentations/Datomic-Database-Value
I think at a larger scale it becomes much harder to try and transform the system around immutable state. At the end of the day if most one needs to do is store CRUD items the easiest way forward is persist schema to a DB.
filesystem is basically a database with simpler interface, less guarantees and more ways to mess up.
internal global mutable state within your program is just a wild west. especially in multi-threaded environments.
Yes.
> If global, mutable state is considered problematic when programming in the small (i.e. single program), why is it unquestioned when building much larger, more complicated systems that we also need to reason about?
It is, and should be, considered problematic. The filesystem can mostly be avoided. The database should probably be rebuilt into something that's easier to reason about. See e.g. https://www.confluent.io/blog/turning-the-database-inside-ou...
It's not unquestioned at all. See for instance, CQRS and event sourcing.
Mutable state is generally a problem, but undelimited mutable state is a serious headache.
Yes, hence file (and database) locking!