I was wondering what would be replacing Excel/VBA after 3 decades as a citizen developer alternative. I could not think of something that even comes close. Any ideas?
Even though it's a visual environment, everything is strongly text based, from HTML to CSS to JSON payloads.
Example: Imagine a CI/CD pipelines using notebooks.
I hate Jenkins/Hudson style build systems so much I could just spit. I just want to run a shell script.
(Alas, I haven't had the gumption to try this idea out yet. Soon.)
These are probably not impossible to solve for notebook-style, but there are not many efforts to solve them or they are not even acknowledged as problems.
Edit: There is Pluto for Julia that attempts to solve the state-problem. I have not used it in practice though; I've given up on Julia, in large part because Julia community tends to be even actively hostile towards "stateless" development.
By "stateless", I'm assuming you mean functional programming paradigms of immutable, idpotent, and no side effects.
FWIW, for build pipelines, my quarter-baked notion is to use ZFS snapshots (or equiv).
I'll check out Pluto for Julia.
As you know, state is a challenge for "serverless" too.
I've been reacquainting w/ RDBMS tools. There are a few new strategies (implementions) for change tracking. Back in the day, we just banged the rocks together (ook, ook), so I'm very eager to learn the new hotness.
Immutability and idempotencency are good, and related, ideals too, although I think these can get too "unergonomic" if taken too dogmatically (like in Haskell or Redux), they should be used with almost goto-level discretion.
Of course there's the clear (short term) usability benefit of maintaining the memory state in that stuff doesn't have to be recomputed. But we can have that benefit and be stateless with pure functions and memoization. I quite often whip up a buggy and brittle ad-hoc solution to do so. There was also the IncPy project [1] that did this more rigorously, but it hasn't been updated in 13 years.
In general I'm a bit baffled why pure function memoization is so rarely used or proposed. Despite the old adage, cache invalidation is not actually half of the three hard problems in CS. With pure functions it's trivial.
Another baffle is why snapshotting/change tracking (and compressing) file systems haven't caught on. Instead these tend to get implemented badly in any sufficiently complicated application.
Also the "higher-level side effects" apply more or less identically to REPL development.
I wasn't even thinking about REPL style work. Mea culpa: I don't actually know how jupyter et al work, so I'm talking out my hat.
Your explanation reminded me of "prevalent" persistence (vs full orthogonal persistence). I guess I assumed something like that was happening between cells.
I suppose it's analogous to the transition of UI frameworks.
Bad: Mutant components directly.
Good: Mutate thru event queue. Get undo/redo for free. Debugging still sucks.
Better: Pretend it's a simulation and use an entity component system. I think this is what the kids are calling "reactive".
> pure function memoization
Answering just for me: because I'm just a simple bear.
I've been imperative for so long, continuations, currying, and lazy eval break my brain. Yup, a fully functional world would be a lot more simple. Maybe it's time for me to revisit clojure.
Thanks again. This is fun to think about.
But this may not be good for anybody in the long run. For example it tends to lead many students to not understand basic concepts, like variables. Which is understandable because variables don't behave like variables in notebooks (e.g. the same variable in the same notebook may refer to different values in different cells depending on how they are run). For many students this can cause almost insurmountably wrong mental models (which they will of course carry to "production" later on).
But as I argued in another thread here, it doesn't have to be this way. E.g. Pluto does notebooks in a more rigorous manner.
Almost all "software engineering" languages and tools makes getting started and actually getting something done quickly needlessly difficult. Probably uncontroversial that git UI is a total mess, and things are getting even worse with more build tools, dogmatic static typing and general pointless ceremony.