Application-as-a-Function Thinking
doridori.github.io
doridori.github.io
Testing becomes so much easier too, as one can instantiate a the whole web routing aspect, without having to bind it to a port and having to send real http requests.
If strongly suggest people to take a look at it. It's not perfect, but it's a lot simpler than other frameworks and libraries. And it's a shift in some of the current mentality of using heavy frameworks (such as spring boot) which blow up anyone's cognitive load.
PP talks about general setup of the server in http4k (which I'm not a fan actually, because server setup is a very small thing done once. Just as a build file, I actually want it as long and verbose as possible).
A functional specification for a stateful system is a function from a list of all inputs to an output, i.e. `fun spec(inputs: List(Input)): Output`.
This kind of specification doesn't need to specify `State` at all, as we can simply refer to previous inputs. E.g. imagine a counter system with the inputs `Increment`, `Reset`, and `Get`. The functional spec for `Get` is "find the latest reset and then count the number of `Increment`s since then".
I think this is neat, because if you are developing the functional spec with a domain expert that doesn't know programming you don't wanna bother them with, for them, unnecessary bookkeeping details.
From this spec you can then figure out what `State` needs to be, but this can be done as a separate step not involving the domain expert but rather other developers.
I learnt this from the Cleanroom software engineering people, they call the spec without `State` a "black box spec", while the one with `State` is called a "state box spec". (There's also a third step called "clear box spec" where they break down the "state box spec" into functions.)
I've been playing around with the idea of designing a specification language that lets you write "black box specs" together with some basic sanity checks of those specs, because I think there's a lot of value in this sort of structure.
For example even if your application doesn't follow the "application-as-function" pattern, you can still use your "state box spec" as an oracle when doing property-based testing of your application.
Hence much easier if the language itself precludes breaking out of the pattern.
> This kind of specification doesn't need to specify `State` at all, as we can simply refer to previous inputs. E.g. imagine a counter system with the inputs `Increment`, `Reset`, and `Get`. The functional spec for `Get` is "find the latest reset and then count the number of `Increment`s since then".
I could be misunderstanding you, but this does not sound functional to me as `spec` seems to have access to previous inputs to `spec` invocations. What am I missing here?
> I think this is neat, because if you are developing the functional spec with a domain expert that doesn't know programming you don't wanna bother them with, for them, unnecessary bookkeeping details.
This sounds very similar to DDD (like as described as the DDD book at the end of the blog post).
> I learnt this from the Cleanroom software engineering people
At IBM direct, or are there any particular resources you can point to which you found useful?
Many thanks for the contribution.
The counter's description is literally Church numbers.
In a stateful system (where outputs depend on previous inputs) you need to have access to all previous inputs in order to determine the output of some input, right? Also keep in mind that this is a specification, not an efficient implementation. If you'd implement it like this you'd have to recompute the output based on the inputs over and over again for each new input. And yeah, you can think of it as some kind of functional composition as nine_k pointed out.
> This sounds very similar to DDD (like as described as the DDD book at the end of the blog post).
Yeah, I think DDD gets this right: some lightweight methods (e.g. event storming) around inputs and outputs. But yeah, Cleanroom did this in the 80s.
> At IBM direct, or are there any particular resources you can point to which you found useful?
On this particular topic, see Harlan Mills' "Stepwise Refinement and Verification in Box-Structured Systems" (1988): https://trace.tennessee.edu/utk_harlan/16/
Flux was before, but it’s not about application as a function
Elm semantics were originally loosely based on functional reactive programming in the sense of Conal Elliot, as per the thesis of its author.
Redux on the other hand is just imperative programming folks rediscovering that a state transition can be described as a pure function S -> S, with all the benefits that come along with it. This is as old as lambda calculus itself. Words like reducer are just red herrings.
Now imagine if you tried to move the state out of all your components into a single location, so that your components can be stateless functions. Now that your variables are out of context and might collide with each other, you might need to rename this `collapsed` state into something like `landingPageItem4AccordianCollapsed`. This is why you shouldn't use global state (like Redux) for everything, and it's often best to keep state close to the UI component it belongs to. This keeps things grouped by context, and makes it easier to navigate and reason about the code.
[1]: https://getbootstrap.com/docs/5.0/components/accordion/
Somebody correct me if I'm wrong, but I believe this is what React is already doing under the hood when you use useState(). React probably has some big internal global state, and then maintains mappings between state and components, so that when a component is re-rendered, React can pull up the corresponding state for that component.
So even though the components are technically stateless, they can still be considered "stateful" because each component has a corresponding portion in this global state. Notice how in your landingPage.Items[].Accordion, if you change the layout of your landing page, you might now have to change the structure of your global state as well. So this is why React's syntax for keeping the state definitions inside the component is so nice. It prevents you from trying to keep the structure of your global state and the layout of your components in sync.
P.S. I like your work on torrents :P I knew I recognized that username from somewhere
IMHO, Component style (React) and State style (Elm) can be interpreted as two styles of isolating (or decoupling), just very orthogonal to each other. Isolating is a very basic necessity for keeping complexity at bay.
Component style isolation works really well for "inner UI state", like your accordion example. But it quickly gets out of hand once you get to some non-UI state that is shared between a bigger component hierarchy.
State/View/Message isolation works really well for actually keeping the views pure and having only one change-message channel. But the state model quickly (and message handler) quickly gets huge. You also cannot publish/provide a small "component" with local state.
Your totally right, it's pretty similar conceptually to Elm in many ways, amazingly I only discovered Elm in the last 8 weeks or so after writing applications in this way for the last few years, maybe not surprisingly as I have been a mobile developer for 13 years and Elm is not big in the mobile space!
> Application is all about transitions of state, yes, but in context.
Like the other commentor says, it would be great to hear more about what you mean exactly by "in context" here.
I am assuming when a user interacts with one control button / dial / interaction it has a cascade effect on other controls / displays etc?
I am also assuming you already need to have a data structure representing the complete state of the system no?
There are multiple interesting aspects here, like are you / do you need to utilise complex state machines here? Just the subject of imperative vs functional state-machines is interesting and a small part of the wider approach.
Another interesting area is it depends on the language and execution environment you are operating in and how expensive operations over immutable data structures are.
Lots of questions!
Sometimes yes, it depends on the instrument. Some have user interaction, but most don't and are just controlled by the software I write. When I said "knobs" I didn't mean literal knobs, just all the different things these gizmos can do .. they come with thousand-page manuals.
OTOH sometimes shit breaks and you have to go in and send commands manually over telnet to unfuck it, or to gracefully shut down the operation without some delicate and expensive piece of equipment getting torn to pieces.
>I am also assuming you already need to have a data structure representing the complete state of the system no?
Nope, it's only feasible to keep track of the most salient variables. The code is very imperative: machine_a.do_this(), machine_b.do_that(), etc, sometimes for hundreds of lines. So the state changes in the background but it's not legible to us. The machines themselves probably have some representation of their own state in their firmware, but there isn't a single omniscient data structure for the whole system.
>There are multiple interesting aspects here, like are you / do you need to utilise complex state machines here? Just the subject of imperative vs functional state-machines is interesting and a small part of the wider approach.
Yeah we use finite state machines to organize our code, but it's very coarse grained. Stuff like: LoadingState, ArmatureIsMovingState, TakingMeasurementState, etc. Each state in itself will issue many different commands to different machines. It isn't really formally defined where the boundaries are, it's just an informal way to orient ourselves so we can draw a flowchart and not lose our minds from the complexity, and so that there's some way of comparing similar-but-different programs (e.g. a run that measures current and a run that measures Young's modulus will both go in a TakingMeasurementState, but obviously the commands sent will be different).
It's not very amenable to automated unit testing ... I'm not sure what that would even mean in this context. The correct behavior you're testing against would just be ... a sequence of commands to the machines. Which is what the code does already. A unit test would be tautological.
>Another interesting area is it depends on the language and execution environment you are operating in and how expensive operations over immutable data structures are.
We're using Python. I know that pyresistent exists, but honestly we could probably get away with doing naive deepcopies for every single operation and it wouldn't matter for performance. The runtime is completely dominated by the latency of waiting for the machines to do physical stuff (no concurrency for the machine-controller code thankfully; we're not insane). Immutable data structures don't really matter if we can't write down the entire state in the first place.
----
I guess the problem is that pretty much everything interesting that the code does is a side-effect. There are a few things here and there that can be factored out to pure functions and tested, but the overwhelming bulk is the "imperative shell". It's like one of those planets where the core has cooled and the crust has frozen down to comprise most of the total mass.
I think you have correctly identified the important difference between the kind of applications I have been writing (mobile applications, which are relatively small and often have a good chunk of both side-effectful code and biz logic) and what your doing, which does not sound like it would have much to gain by spitting into a core & shell.
I have not much to offer you, having not worked on code such as you describe.
Applications are not functions.
How many applications that you use on a daily basis work as follows:
1. You prepare some parameters
2. You start the application with those parameters.
3. The application goes away and thinks for a bit.
4. The application returns with a result and then exits.
Trying to make actual applications and system fit into the function (or procedure) mold is, IMHO, one of the biggest obstacles to software simplicity, as there is a fundamental architectural mismatch here.
In the early days of computing, a lot of programs actually did work this way, which is why DSLs for algorithms (ALGOL) were appropriate, and probably where the idea originated that they are actually general purpose languages. Which they are not.
It is actually my full time job to support a suite of applications that do exactly this.
However, whereas they used to make up the overwhelming majority of programs, they are now minority, yet we still try to model all programs like this.
Which is obviously possible, after all we're doing it. But it's not a good idea.
Fundamentally an application can be written to have at it’s core, a single, stateless pure function, behold!
https://repository.upenn.edu/library_papers/68/
https://rd.springer.com/content/pdf/10.1007/978-3-540-92698-...
TFA basically describes Elm’s architecture https://guide.elm-lang.org/architecture/ which is one of the state of the art abstractions for building applications where you model state changes in a central function.
Or watch Gary Bernhardt's "Functional core, imperative shell" talk (https://www.destroyallsoftware.com/screencasts/catalog/funct...) that compelled TFA.
Here is a quote from the article:
Fundamentally an application can be written to have at it’s core, a single, stateless pure function, behold!
And my point is that, yes, you can do that, but you shouldn't.
You haven't explained what is wrong with using a pure function to model state changes in an application. To do so, I think you'd also have to contend with the fact that a solution like Elm works quite well in practice.
Web app abstraction frameworks like Rack (Ruby), WAI (Haskell) and many others work exactly like that: they allow you to supply them with a function taking a HTTP request and returning a HTTP response, then run that for each incoming request. Nothing magical about that and it works extremely well in practice.
So yeah, we can say web servers work like functions if we cut out all the bits that don't and call them separate applications. But realistically they're not.
No Rails (or other) app has a function like this where the entire contents of the DB are passed in as a parameter.
And you can't handwave complex internal state as simply another parameter to a function because that complex state can change. That makes the function no longer solely dependent on the parameters passed in. Its behavior is dependent on the parameters passed in plus any changes made by other processes to the complex internal state.
Sure, it is possible to model these things as functions, for example by "passing" the entire world as a "parameter". However, it is an extremely contrived modelling, basically you're hammering the square problem domain into the round function hole with the biggest hammer you can find until it "fits".
It is also possible to model these things as Turing machines, or NAND gates...
Moat interactive applications can be seen similarly, they just do that in a loop, for every event coming from the user.
I blame myself for this misunderstanding as I wrote the article. I certainly am not suggesting that an application is a function, rather an application can be represented by single function as its core.
There is an important distinction here as the production applications I have utilised this architecture in certainly do not literally behave as functions to the external user in the manner you describe, the user interaction is in no way affected by the architectural choice.
As others have eluded to this core pure function is really just a boundary between the external side-effect laden world (e.g. UI / network / sensors / OS etc) and the applications business / domain logic and NOT the boundary between the OS and the application which seems to be what you are describing.
I appreciate your input and the discussion it is provoking.
Like take the log out action as an example. Mid request we are going to have to change the state from a user logged in to a user logged out. Otherwise when we render the home screen, as an example, they'll see the logged in version instead of the anonymous user one.
And realistically we want need a transition state to show a successfully logged out message since we do not want that message to show the next time they come. So the whole sequence for this simple action is:
Old State -> Intermediate State -> Render Response -> New State for the next request.
So we have to change state a couple times and pass that along.
And then how do we deal with stuff that's actually state. Like a database or whatever. Are we really passing that as a function parameter so that 10 calls deep can use it?
And what happens 6 months in when we want a caching layer? Do we go back and edit every single function to accept a Redis argument now?
Realistically, no. We have some sort of config or server object that we pass along. So we have an object that contains all of our server state that gets passed to most functions. How is that better than a single global variable holding that state accessible everywhere in the application?
> ... as a thought experiment it makes sense but it's not transferrable to the real world.
For the record, it's certainly not a thought experiment, I have been using this in the real world for a few years.
> Like take the log out action as an example...
The problem you outline with regards to intermediate UI states around the logout flow example is one of assumed responsibilities. If an application needs to show some specific transient UI when the application has moved between two states well, that's a UI concern and does not have to be modelled as part of the core application state. For example if you have a UI for an application which is required to show some jazzy animation when a user transitions from logged-in to logged-out that in no way needs to be part of the core application logic. The UI can just inspect the incoming state and conditionally when detecting a move from LoggedIn to LoggedOut trigger some UI animation / transient dialog / whatever.
One nice thought experiment / thought pump I find of value when dividing core application (~domain) and UI responsibilities is thinking about if you were to switch the UI from say a mobile framework (say Android/Compose) to a desktop terminal, what stuff stays the same and what stuff is display specific. In the above example you most likely would not want to show a jazzy animation on the terminal and therefore is display specific and should no be modelled in the core application logic. This is a nice simple thought process for dividing these responsibilities.
> And then how do we deal with stuff that's actually state. Like a database or whatever. Are we really passing that as a function parameter so that 10 calls deep can use it?
No, a database would fall under IO and would make more sense to access behind a Command interface in this pattern, like most other side-effect interactions. This fits into the "functional core imperative shell" mindset referred to in the post.
> And what happens 6 months in when we want a caching layer? Do we go back and edit every single function to accept a Redis argument now? Realistically...
I am in no way experienced with BE caching layers, my experience is mostly client side mobile applications. However, my first approach for an in-memory DB cache would be to chuck that in front of the DB access behind the same abstraction, and also this would be behind the Command interface, again like all other side-effect interactions.
And apologies for the misunderstanding due to my terseness, which is partly due to me having written about this in great depth on a number of occasions, here are two examples:
- Why Architecture-Oriented Programming Matters[1]
- Can Programmers Escape the Gentle Tyranny of call/return?[2]
The basic point is that we are so used to programming simply being call/return (and functional is a subset of that), that it is very hard for us to conceive of programming being anything else.
And one consequence of that is that we try to map every problem onto a call/return (incl. FP) solution, no matter how inappropriate the mapping, and the mapping very often is inappropriate, leading to architectural mismatch. [3][4]
We do this partly because we really don't know better, but also partly because there are tangible benefits, primarily that once we have done that mapping, we can then express whatever we arrived at pretty directly in our languages, because our languages effectively only allow call/return based abstraction.
It's a bit of a conundrum.
[1] https://blog.metaobject.com/2019/02/why-architecture-oriente...
[2] https://2020.programming-conference.org/details/salon-2020-p...
[3] https://repository.upenn.edu/library_papers/68/
[4] https://rd.springer.com/content/pdf/10.1007/978-3-540-92698-...
Most of them, but then I use the CLI a lot. E.g.:
find <params> | grep <params> | xargs foo <params>
- - - -
How do you mean, ALGOL isn't a general purpose language?
2. But yes, programs that act like functions definitely do exist. They even used to be in the majority.
3. ALGOL stands for ALGOrithmic Language. It's a DSL for the domain of algorithms. As are virtually all our mainstream so-called "general purpose" languages.
#2 so how does that modify your original comment, if at all?
#3 What would you contrast our mainstream so-called "general purpose" languages with?
> For me the difficult part was where the side effects fit in and how they're processed, what you call "commands" and "command handler"
Do you mean difficult for you when designing, or difficult to understand in my post?
It has lead to a very modular and maintainable app that has been very easy to test without ridiculous mocking you usually see when needing to interact with object-oriented / imperative frameworks like Cocoa Touch.
With this approach: 1. What downsides have you encountered? 2. How easy do you find it to onboard people? 3. Does Android do the same at your place?
Our next big effort will be fitting the architecture into SwiftUI, which I think will actually be also straightforward since SwiftUI is extremely functional.
2. Onboarding is probably the biggest “downside”. I would say there is absolutely a learning curve that requires some investment but everyone I speak to about it has been pretty positive after getting over that hurdle. And it feels like our productivity is very high.
3. Android does not follow the same pattern. I am planning on sharing your article with the team though. :-)
1. Create a struct (value type in Swift) that implements a protocol (interface) to serve as your ViewModel 2. The main methods to implement are both called “update”. One is called automatically by our infrastructure when Feature state (state that exists across screens) changes. The other is called by the developer in their view controllers to pass along user actions (or other UIKit events). 3. The developer subscribed to all VM changes and updates the view in response. This subscription is done via Combine which is Apple’s FRP framework.
I am planning on writing more about this and the rest of our architecture in the future, hopefully.
Forgive me if I’m oversimplifying, but isn’t this just “stateless servers” and “router-service separation” at its core? Maybe this isn’t targeted for the API crowd, but anybody who has spent even a few months learning modern client-server programming understands this, and web dev is pretty common these days…
Not targeted at the API crowd really no, more client side applications, but it seems similar principles have been applied in various places / frameworks as spoken about in other comments here.
> Maybe this isn’t targeted for the API crowd, but anybody who has spent even a few months learning modern client-server programming understands this, and web dev is pretty common these days
Yes, I'm sure your right, but there are a huge number of people who are not web devs who have not been exposed to these fundamental principles and how they may apply in other operating environments :)
Disclaimer, I've avoided using reactive approaches as much as I can on the server side as it increases the complexity (and cognitive load) and makes diagnosing issues harder. This is my experience in the JVM world, I'm not sure how it is in other languages/platforms.
[This article](https://netflixtechblog.com/zuul-2-the-netflix-journey-to-as...) from Netflix mentions the tradeoffs they experienced when they re-engineered their API gateway to be reactive. I wonder if this is still valid as the article was written 6 years ago.
In my experience the commonality found between client side mobile applications and reactive principles is essentially everything is a stream, and the UI subscribes to those streams.
The approach outlined in the post is similar in terms of the UI observes a (single) stream of data, which can just be a simple (State) -> Unit) as opposed to some reactive library stream primitive.
The approach outlines is different from all the "reactive" client side applications I have seen as there is no proliferation of stream primitives present throughout the codebase, which often seems to create complex code with very little if any user benefit.
I'm not sure what other elements you see in the post which also fall into the "reactive paradigm"?
All of this being on-chain basically gives 'event logging' out of the box so all functions (i.e. contracts) on a chain can be monitored by any developer involved.
(Building this part myself: https://thestackreport.xyz/dashboards/tezos)
> event logging' out of the box so all functions (i.e. contracts) on a chain can be monitored by any developer involved
You cannot be serious..