Simplify your code: Functional core, imperative shell
testing.googleblog.com
testing.googleblog.com
I actually remember early in my career working for a small engineering/manufacturing prototyping firm which did its own software, there was a senior developer there who didn't speak very good English but he kept insisting that the "Business layer" should be on top. How right he was. I couldn't imagine how much wisdom and experience was packed in such simple, malformed sentences. Nothing else matters really. Functional vs imperative is a very minor point IMO, mostly a distraction.
That goes against every bit of advice and training I've ever gotten, not to mention my experience designing, testing, and implementing APIs. Business logic belongs in the data model because of course the rules for doing things go with the things they operate on. API endpoints should limit themselves to access control, serialization, and validation/deserialization. Business logic in the endpoint handler—or worse, in the user interface—mixes up concerns in ways that are difficult to validate and maintain.
Probably many reasons for this, but what I've seen often is that once the code base has been degraded, it's a slippery slope downhill after that.
Adding functionality often requires more hacks. The alternative is to fix the mess, but that's not part of the task at hand.
Another factor, and perhaps the key factor, is that contrary to OP's extraordinary claim there is no such thing as objectively good code, or one single and true way of writing good code.
The crispest definition of "good code" is that it's not obviously bad code from a specific point of view. But points of view are also subjective.
Take for example domain-driven design. There are a myriad of books claiming it's an effective way to generate "good code". However, DDD has a strong object-oriented core, to the extent it's nearly a purist OO approach. But here we are, seeing claims that the core must be functional.
If OP's strong opinion on "good code" is so clear and obvious, why are there such critical disagreements at such a fundamental levels? Is everyone in the world wrong, and OP is the poor martyr that is cursed with being the only soul in the whole world who even knows what "good code" is?
Let's face it: the reason there is no such thing as "good code" is that opinionated people making claims such as OP's are actually passing off "good code" claims as proxy's for their own subjective and unverified personal taste. In a room full of developers, if you throw a rock at a random direction you're bound to hit one or two of these messiahs, and neither of them agrees on what good code is.
Hearing people like OP comment on "good code" is like hearing people comment on how their regional cuisine is the true definition of "good food".
Really?
DDD is described in terms of OOP, but really imo it makes far more sense in fp contexts.
The original 2003 DDD book is very 2003 in that it is mired in object orientation to the point of frequently referencing object databases¹ as a state-of-the-art storage layer.
However, the underlying ideas are not strongly married to object orientation and they fit quite nicely in a functional paradigm. In fact, ideas like the entity/value object distinction are rather functional in and of themselves, and well-suited to FCIS.
Irrelevant, as a) that's just your own personal and very subjective opinion, b) DDD is extensively documented as the one true way to write "good code", which means that by posting your comment you are unwittingly proving the point.
> However, the underlying ideas are not strongly married to object orientation and they fit quite nicely in a functional paradigm.
"Underlying ideas" means cherry-picking opinions that suit your fancy while ignoring those that don't.
The criticism on anemic domain models, which are elevated to the status of anti-pattern, is more than enough to reject any claim on how functional programming is compatible with DDD.
And that's perfectly fine. Not being DDD is not a flaw or a problem. It just means it's something other than DDD.
But the point that this proves is that there is no one true way of producing "good code". There is no single recipe. Anyone who makes this sort of claim is either both very naive and clueless, or is invested in enforcing personal tastes and opinions as laws of nature.
Yes, that is how terminology evolves to not meet a rigid definition that was defined in a different era of best-practice coding beliefs. I'll admit I had trouble mapping the DDD OO concepts from the original book(s) to systems I work on now, but there are more recent resources that use the spirit of DDD, Domain Separation, and Domain Modeling outside of OO contexts. You're right in that there is no single recipe - take the good ideas and practices from DDD and apply it as appropriate.
And if the response is "that's not DDD", well you're fighting uphill against others that have co-opted the buzzword as well.
- https://learn.microsoft.com/en-us/dotnet/architecture/micros... - https://www.infoq.com/news/2013/06/actor-model-ddd/
Yes? And it's just your personal, subjective opinion that this is irrelevant. Most meaningful judgments are subjective. Get used to it.
> DDD is extensively documented as the one true way to write "good code"
Who said this? I've seen it described as a good way to write code, and as a way of avoiding problems that can crop up in other styles. But never as the only way to write good code.
> "Underlying ideas" means cherry-picking opinions that suit your fancy while ignoring those that don't.
No it doesn't. What? The only way I can make sense of what you're saying is if you're cynical toward the very concept of analyzing ideas, which is perhaps the most anti-intellectual stance I can imagine.
> The criticism on anemic domain models [...] is more than enough to reject any claim on how functional programming is compatible with DDD.
Why would an author's criticism of a certain style of OOP make a methodology they have written about incompatible with non-OOP paradigms? That's like saying that it's impossible to make strawberry ice cream because the person who invented ice cream hates strawberries.
> But the point that this proves is that there is no one true way of producing "good code".
There's no "one true way" to build a "good bridge," but that doesn't mean bridge design is all a matter of taste. Suspension bridges can carry a lot more than beam bridges; if you want to drive 18-wheelers across a wide river, a beam bridge will collapse, while a suspension bridge will probably be "good."
I still find myself debating this internally, but one objective metric is how smoothly my longer PTOs go:
The only times I haven’t received a single emergency call were when I left teammates a a large and extremely specific set of shell scripts and/or executables that do exactly one thing. No configs, no args/opts (or ridiculously minimal), each named something like run-config-a-for-client-x-with-dataset-3.ps1 that took care of everything for one task I knew they’d need. Just double click this file when you get the new dataset, or clone/rename it and tweak line #8 if you need to run it for a new client, that kind of thing.
Looking inside the scripts/programs looks like the opposite of all of the DRY or any similar principles I’ve been taught (save for KISS and others similarly simplistic)
But the result speaks for itself. The further I go down that excessively basic path, the more people can get work done without me online, and I get to enjoy PTO. Anytime i make a slick flexible utility with pretty code and docs, I get the “any chance you could hop on?” text. Put the slick stuff in the core libraries and keep the executables dumb
My favorite are things where security policy mandates something like private networking and RBAC, and certain resources only have meaning in those contexts, for heavens sake why are we making their basic args like “enforce_tls” or “assign_public_ip” or “enable_rbac” into variable params for the user to figure out
...
> Coupling across layers invites trouble (e.g. encoding business logic with “intuitive” names reflecting transient understanding). When requirements shift (features, regulations), library maintainers introduce breaking changes or new processor architectures appear, our stable foundations, complected with faster-moving parts, still crack!
https://alexalejandre.com/programming/coupling-language-and-...
Your advice is the opposite of "functional core, imperative shell". The FCIS principle has IS which is generic, to be simple, because it's usually hard to test (it deals with resources and external dependencies). So by being simple, it's more unit testable.
On the other hand, FC is where the business logic lives, which can be complex and specific. The reason why you want that "functional" (really just another name for "composable from small blocks") is because it can be tested for validity without external dependencies.
So the IS shields you from technicalities of external dependencies, like what kind of quirks your DB has, or are we sending data over network or writing to the file, or does the user inputs comands in spanish or english, do you display the green square or blue triangle to indicate the report is ready, etc.
On the other hand, FC deals with the actual business logic (what you want to do), which can be both generic and specific. These are just different types of building blocks (we call them functions) living in the FC.
FCIS is exemplified by user-shell interaction. The user (FC) dictates the commands and interprets the output, according to her "business needs". While the shell (IS) simply runs the commands, without any questions of their purpose. It's not the job of IS to verify or handle user errors caused by wrong commands.
But the user doesn't do stuff on her own; you could take her to a pub and she would tell you the same sequence of commands when facing the same situation. In that sense, the user is "functional" - independent on the actual state of the computer system, like the return value of a mathematical function is only dependent on the arguments.
Another example is MVC, where M is the FC and VC is the IS. Although it's not always exactly like that, for variety of reasons.
You can think of IS as a translator to a different language, understood by "the other systems", while the FC is there to supply what is actually being communicated.
"Functional core, imperative shell" (FCIS) is a matter of implementing individual software components that need to engage with side-effects --- that is, they have some impact on some external resources. Rather than threading representations of the external resources throughout the implementation, FCIS tells us to expel those concerns to the boundary. This makes the bulk of the component easier to reason about, being concerned with pure values and mere descriptions of effects, and minimizes the amount of code that must deal with actual effects (i.e. turning descriptions of effects into actual effects). It's a matter of comprehensibility and testability, which I'll clumsily categorize as "verification": "Does it do what it's supposed to do?"
"Generic core, specific shell" (GCSS) is a matter of addressing needs in context. The problems we need solved will shift over time; rather than throwing away a solution and re-solving the new problem from scratch, we'd prefer to only change the parts that need changing. GCSS tells us we shouldn't simply solve the one and only problem in front of us; we should use our eyes and ears and human brains to understand the context in which that problem exists. We should produce a generic core that can be applied to a family of related problems, and adapt that to our specific problem at any specific time using a, yes, specific shell. It's a matter of adaptability and solving the right problem, which I'll clumsily categorize as "validation": "Is what it's supposed to do what we actually need it to do?"
Ideally, GCSS is applied recursively: a specific shell may adapt an only slightly more generic core, which then decomposes into a smaller handful of problems that are themselves implemented with GCSS. When business needs change in a way that the outermost "generic core" can't cover, odds are still good that some (or all) of its components can still be applied in solving the new top-level problem. FCIS isn't really amenable to the same recursion.
Both verification and validation activities are necessary. One is a matter of internal consistency within the component; the other is a matter of external consistency relative to the context the component is being used in. FCIS and GCSS advise on how to address each concern in turn.
I have considered them being orthogonal, but then the definition of the "shell" and "core" becomes problematic in this comparison. What you call shell in GCSS is not shell in FCIS at all, more like a boundary. Even there it is questionable whether boundary should be more specific than the core. At the core, things can be more integrated than at the boundary, and so it can have more business-specific logic.
The definition question is, if you take an application, where is the business logic, is it in the "core" or not? I would say it is, literally what the application's main purpose is its "core". And "shell" is similarly well-defined. For example, UI without an engine implementing the actual logic is just a "shell".
I am not disputing GP's advice as you understand it, although I feel it is perhaps a little bit simplistic if not tautological ("prefer generic building blocks where possible"), and really muddles up what the core and shell is in the FCIS meaning.
I agree -- if you're trying to make the words "shell" and "core" mean the same things between FCIS and GCSS, or identify the same parts of a program, then there will be problems. I think FCIS and GCSS are just two different ways of analyzing a program into pieces. Just as the number 8 can be seen through addition as 3 + 8 and through multiplication as 2 * 4, a program can be analyzed in multiple ways. If you view a program through the lens of FCIS, you expect to see a broad region of the program in which side-effects don't occur and a narrow region in which they do. If you view a program through the lens of GCSS, you expect to see broad parts of the program that solve general problems, and narrower regions in which those problems are instantiated in specific. The narrower regions are all "shell"-shaped, but that doesn't mean they are "the" shell. They have in common simply that they wrap a bulk of functionality to interface it to a larger context.
> At the core, things can be more integrated than at the boundary, and so it can have more business-specific rules.
I tend to disagree. Decomposition is a fundamental part of software engineering: we decompose a large problem into smaller ones, solve those, them compose those solutions into a solution to the large problem (c.f. Parnas' "On the criteria to be used in decomposing systems into modules"). It is often easier to solve a more general problem than the one originally given (Polya's principle). Combining the two yields GCSS.
A solution to each individual small problem can be construed as having its own generic core, and the principles used in composing sibling solutions constitute the specific shells that wrap them, allow them to interface, and together implement a solution to a higher-level problem.
Because there are multiple of these "cores", each solving a decomposed part of the top-level problem, it's hard for me to see how "At the core, things can be more integrated than at the boundary".
> The definition question is, if you take an application, where is the business logic, is it in the "core" or not?
I don't mean to be a sophist, but I think I need a more precise meaning of "business logic" before I can answer this question. In the process of solving successively smaller (and more general) problems, we abstract away from the totality of the business problem being solved, and address smaller and less-specific aspects of that problem. It may be that each subproblem's solution is architected as an individual instance of FCIS, as is often argued for microservice architectures; or that each subproblem is purely functional and only the top-level solution is wrapped in an imperative core; or anywhere in between. Needless to say, I think that choice is orthogonal.
As a result, I would say that the business logic itself has been factorized and distributed across the many subproblems and their solutions, and that indeed the "specific shell"s that are responsible for specializing solutions toward the specific case of the business need may necessarily include business logic. For instance, when automating a business process, one often needs to perform a complex step A before a complex step B. While both A and B might be independently solvable, orchestrating them together is still "business logic", because they need to be performed in order according to business needs.
(In all of this, perhaps you can see why I don't think the "core" and "shell" of FCIS should be identified with the "core" and "shell" of GCSS. Words are allowed to have contextual meanings!)
Sure, but this discussion is about FCIS, that's the context, and the GP should consider that.
" think I need a more precise meaning of "business logic" before I can answer this question"
Well, some examples. A tax application - the tax calculation according to the law. A word processor - layouting and rendering engine. A video game - something that calculates the state of the world, according to the rules of the game.
So a game is a good example where the core can be more specialized than the shell. You can imagine a generic UI library shared by a bunch of games, but a generic game rules engine - that's just a programming language.
"Decomposition is a fundamental part of software engineering: we decompose a large problem into smaller ones, solve those, them compose those solutions into a solution to the large problem"
There is a big misconception in SW engineering that the above decomposition always exists in a meaningful way. Take the tax calculation for example. That cannot be decomposed into pieces that are generic, and potentially reusable elsewhere. It's just a list of rules and exceptions that need to be implemented as stated. You can decompose it into "1st part of calculation" and "2nd part of calculation", but that's meaningless (unhelpful). (Similarly for the game example above, the rules only exist in the context of other rules.)
Surprisingly many problems are like that, and that makes them kinda difficult to test.
Quite right! However, the tax code does change with some regularity, and we can expect that companies like Intuit should have gotten quite good by now -- even on a pure profit motive -- at making it possible to relatively quickly modify only the parts of their products that require updating to the latest tax code. To put it another way, while it might be the case that the tax code for any given year is not amenable to decomposition, all tax codes within a certain span of years might be specific instances of a more general problem. (I recall a POPL keynote some years back that argued for formalizing tax codes in terms of default logic!) By solving that general problem, you can instantiate your solution on the given year's tax code without needing to recreate the entire program from scratch.
To be clear, I'm the one who brought subproblem decomposition into the mix, and we shouldn't tar the top-level commenter with that brush unnecessarily. Of course some problems will be un-decomposable "leaves". I believe their original point, about a specific business layer sitting on top of a more general core, still applies.
> So a game is a good example where the core can be more specialized than the shell. You can imagine a generic UI library shared by a bunch of games, but a generic game rules engine - that's just a programming language.
As it happens, the "ECS pattern" (Entity, Component, and System) is often considered to be a pretty good way of conceptualizing the rules of a game. An ECS framework solves the general problem (of associating components to entities and executing the systems that act over them), and a game developer adapts an ECS framework to their specific needs. The value in this arrangement is precisely that, as the game evolves and takes shape, only the logic specific to the game needs to be changed. The underlying ECS framework is on the whole just as appropriate for one game as for any other.
(I could also make a broader point about game engines like Unity and Unreal, and how so many games these days take the "general" problem solved by these engines and adapt them to the "specific" problem of their particular game. In general, nobody particularly wants to make engine-level changes for each experiment during the development of a game, even though sometimes a particular concept for a game demands a new engine.)
> Sure, but this discussion is about FCIS, that's the context, and the GP should consider that.
I understood the original commenter as criticizing FCIS (or at least the original post, as "grasping for straws") and suggesting that GCSS is generally more appropriate. In that context, I think it's natural to interpret their use of "core" and "shell" as competing rather than concordant with FCIS.
As someone who does this himself for taxes, you're looking only at the "specific shell" part. The generic core is the thing that does the math - spreadsheet, database, whatever. The tax rules are then imposed on top of that core.
The example on the homepage is the "specific shell" - simple and easy to use, and by far the most common usage, but if you scroll down the table of contents on the API page (https://requests.readthedocs.io/en/latest/api/) you'll see sections titled "Lower-Level Classes" and "Lower-Lower-Level Classes" - that's the generic core, which the upper level is implemented in terms of.
* opinions vary
Because Haskell programs pretty much have to be FCIS or they won't compile.
How it plays out is...
1. A Haskell program executes side effects (known as `IO` in Haskell). The type of the `main` program is `IO ()`, meaning it does some IO and doesn't return a value - a program is not a function
2. A Haskell program (code with type `IO`) can call functions. But since functions are pure in Haskell, they can't call code in `IO`.
3. This doesn't actually restrict what you can do but it does influence how you write your code. There are a variety of patterns that weren't well understood until the 1990s or later that enable it. For example, a pure Haskell function can calculate an effectful program to execute. Or it can map a pure function in a side-effecting context. Or it can pipe pure values to a side-effecting stream.
Recently, I did a major python refactoring project, converting a prototype/hack/experiment into a production-quality system. The prototype heavily intermixed IO and app logic, and I needed to write unit tests for the production system. Even with fixtures and mocking, unit testing was painful and laborious, so our test coverage was lousy.
Partitioning the core classes into pure and impure components was the big win. Unit testing became trivial and we caught lots of bugs in the original business logic. More recently, we changed the IO from files to a DB and having encapsulated the IO was also a win.
Splitting IO and pure code was just routine refactoring, not a full redesign. Our app logic wasn't strictly pure because it generates pseudorandom numbers and logs events, but practically speaking, splitting the IO and shell from the relatively pure app logic made for much cleaner code.
In retrospect, I consider FCIS a good practice that I first learned with Haskell. It's valuable elsewhere, even when used in a less formal way than Haskell mandates.
This violates KISS and YAGNI and potentially leads to overengineering code and excessive abstraction
That's the gotcha though. Everything applied with due care and cognizance works. This is not what is being discussed here. What the author suggests does lead to overengineering though. Think of stereotypical enterprise Java code if you need examples
Even though the business currently doesn't have a need to e.g. support any other currency than USD and EUR, an experienced developer will clearly see that it is unlikely to stay that way for long, so doing some preliminary preparation for generalizing currencies may well worth the time.
Your regular business requirements are way more complex than just a currency list. This is like trying to justify your point using an oversimplified example imo.
You look at electronics and vehicles, the components inside are generic, many devices use exactly the same internal components... All the internal components are chosen specifically for their ability to handle a wide range of conditions (pressure, heat, electromagnetic interference) and also based on how broadly compatible they are with other components and tools... But somehow, when it comes to software, we treat it as if it's completely different.
There is a lot of value in using generic components which can handle a wide range of use cases. You want to avoid changing dependencies as much as possible because it takes time and effort to write robust code. You want a solid foundation which can solve a well defined (but not necessarily narrow) range of problems. Some modules can be used to solve many different problems, in completely different business domains but they may have a very simple, well defined interface. Think of a screw... Very simple, well-defined interface, can be applied to a huge range of use cases.
> Nothing else matters really. Functional vs imperative is a very minor point IMO, mostly a distraction.
I'm torn on this. This really is the faster way to higher quality.
OTOH, if more developers knew this, I wouldn't be so much more faster when I create my systems for clients. I'd just be a "normal 1x dev".
I like implementing features, sans AI-assistance, in my LoB applications faster than devs with Claude code doing so on their $FRAMEWORK.
I think this is a very common mistake. You've spent years, maybe decades, writing code and now you want to magically transfer all that experience in a few succinct articles. But no advice that you give about "the correct philosophy" is going to instantly transfer enough knowledge to make all large companies write good code, if only they followed it. Instead, I'm sure it's valuable advice, but more along the lines of a fragment within a single day of learning for a diligent developer.
A company I worked recently had a more extreme version of this mistake. It had software written in the 1980s based on a development process by Michael Jackson (no, not that one!), a software researcher that had spent his whole career trying to come up with silly processes that were meant to fix software development once and for all; he wrote whole books about it. I remember reading a recent interview with him where he mourns that developers today are interested in new programming languages but not development methodologies. (The code base I worked on was fine by the way, given that it was 40 years old, but not really because of this Jackson stuff.)
I'm reminded of the Joel on Software article [1] where he compares talented (naturally or through experience) developers as being like really talented expert chefs, and those following some methodology as being like people working at McDonald's.
[1] https://www.joelonsoftware.com/2001/01/18/big-macs-vs-the-na...
Good old "Programming as Theory Building". It's almost impossible to achieve this kind of transfer without already having the requisite lived experience.
Business layers should be accessible via an explicit interface/shape that is agnostic to the layers above it. So if the org decides to move from mailchimp to some other email provider the business logic can remain untouched and you just need to write some code mapping the new provider to the business logic's interface.
Maybe our visualizations are mixed up, but I always viewed things like cloud providers, libraries etc. as potentially short lived whereas the core logic could stick around forever.
For concerns of code complexity and verification, code that asks a question and code that acts on the answers should be separated. Asking can be done as pure code, and if done as such, only ever needs unit tests. The doing is the imperative part, and it requires much slower tests that are much more expensive to evolve with your changing requirements and system design.
The one place this advice falls down is security - having functions that do things without verifying preconditions are exploitable, and they are easy to accidentally expose to third party code through the addition of subsequent features, even if initially they are unreachable. Sun biffed this way a couple of times with Java.
But for non crosscutting concerns this advice can also be a step toward FC/IS, both in structuring the code and acclimating devs to the paradigm. Because you can start extracting pure code sections in place.
> having functions that do things without verifying preconditions are exploitable
Why would you do this? The separation between commands and queries does not mean that executing a command must succeed. It can still fail. Put queries inside the commands (but do not return the query results, that's the job of the query itself) and branch based on the results. After executing a command which may fail, you can follow it with a query to see if it succeeded and, if not, why not.
https://en.wikipedia.org/wiki/Command%E2%80%93query_separati...
Performance and re-use are two possible reasons.
You may have a command sub-routine that is used by multiple higher-level commands, or even called multiple times within by a higher-level command. If the validation lives in the subroutine, that validation will be called multiple times, even when it only needs to be called once.
So you are forced to choose either efficiency or the security of colocating validation, which makes it impossible to call the sub-routine with unvalidated input.
hinkley poses this as a fault in CQS, but CQS does not require your commands to always succeed. Command-Query Separation means your queries return values, but produce no effects, and your commands produce effects, but return no values. Nothing in that requires you to have a command which always succeeds or commands which don't make use of queries (queries cannot make use of commands, though). So a better question than what I originally posed:
My "Why would you do this?" is better expanded to: Why would you use CQS in a way that makes your system less secure (or safe or whatever) when CQS doesn't actually require that?
CQS will rely on composition to do any If A Then B work, rather than entangling the two. Nothing forces composition except information hiding. So if you get your interface wrong someone can skip over a query that is meant to short circuit the command. The constraint system in Eiffel I don’t think is up to providing that sort of protection on its own (and the examples I was given very much assumed not). Elixir’s might end up better, but not by a transformative degree. And it remains to be seen how legible that code will be seen as by posterity.
> The one place this advice falls down is security - having functions that do things without verifying preconditions are exploitable
My understanding of your comment was that "this advice" is CQS. So you're saying that CQS commands do not verify preconditions and that this is a weakness in CQS, in particular.
Where did you get the idea that CQS commands don't verify preconditions? I've never seen anything in any discussion of it, including my (admittedly 20 years ago) study of Eiffel.
If A then B()
Versus
B()
Somewhere there’s a B without the associated query. Call it what you want, at the bottom of the tree two roads diverge. Otherwise there is no Separation in your CQS.
ETA: once you get down to the mutation point you aren’t just dealing with immutable data. You’re moving things around, often plural.
You don't need the command to return anything (though it can be more efficient or convenient). It can set state indicating, "Hey, I was called but by the time I tried to do the thing the world and had changed and I couldn't. Try using a lock next time."
if (query(?)) {
command(x)
result := status(x) // ShouldHaveUsedALockError
}
I’d argue that the separation makes things worse here, because it creates additional hidden state.
Also, as I stated, this is not about error handling.
(One might argue that all RPC is asynchronous; all such arguments eventually lead to message buses, at-least-once delivery, and the reply-queue pattern, but maybe that's also just presentation fodder.)
Caches always mess up computational models because they turn all reads into writes. Which makes things you could say with static analysis no longer true. I know a lot of tricks for making systems faster and I’ve hardly ever seen anyone apply most of them to systems after caching was introduced. It has one upside and dozens of downsides as bad or worse than this one.
If you really need locks, that generally locks you out of this kind of architecture, which makes the CQRS value proposition much flimsier.
What’s being described here is something lower level, that you keep as much code as you can as a side-effect-free “pure functional core”. That pattern is useful both for the “command” and “query” side of a CQRS system, and is not the same thing as CQRS
Query and ask are synonyms and represent the same idea in this context.
The key design goal in this thread was to create a pure functional core, which you can “ask” things of. That pattern is useful on both the command and query side of a CQRS system, and a different thing from splitting up mutating and reading operations as CQRS proposes
Maybe I misunderstand you though. Say you have a CQRS system that reads and writes to a database. Are you proposing the query side be implemented in pure side-effect-free functional code? How should the pure code make the network calls to the database?
But there’s nothing in the more general idea of “separate reads from writes” that mandates “no validation on writes”
https://hemath.dev/blog/command-query-separation/
Down at the bottom it gets into composition to make utility functions that compose several operations. Any OO system has to be careful not to expose methods that should have been private, so that’s not specific to CQS. It’s just that the opportunities to get it wrong increase and the consequences are higher.
email.bulkSend(generateExpiryEmails(getExpiredUsers(db.getUsers(), Date.now())));
Many times, it has confused my co-workers when an error creeps in in regards to where is the error happening and why? Of course, this could just be because I have always worked with low effort co-workers, hard to say.
I have to wonder if programming should have kept pascals distinction between functions that only return one thing and procedures that go off and manipulate other things and do not give a return value.
db.getUsers()
|> getExpiredUsers(Date.now())
|> generateExpiryEmails()
|> email.bulkSend()
bulk_send(
generate_expiry_email(user)
for user in db.getUsers()
if is_expired(user, date.now())
)
Writing custom monad syntax is definitely quite a nice benefit of functional languages IMO.
Users.all()
|> Enum.filter(&Users.is_expired?(&1, Date.utc_today()))
|> Enum.map(&generate_expiry_email/1)
|> tap(&IO.inspect(label: "Expiry Email"))
|> Enum.reject(&is_nil/1)
|> bulk_send()
It just becomes so easy to incrementally add functionality here.
---
Quick syntax reference for anyone reading:
- Pipelines apply the previous result as the first argument of the next function
- The `/1` after a function name indicates the arity, since Elixir supports multiple dispatch
- `&fun/1` expands to `fn arg -> fun(arg) end`
- `&fun(&1, "something")` expands to `fn arg -> fun(arg, "something") end`
clock in this case is a thing that was supplied to the class or function. It could just be a function: () -> Instant.
(Setting a global mock clock is too evil, so don't suggest that!)
This is why we have tests which we need to update every 3 months, because somebody said this. This is of course, after a ton of research went into finding out why the heck our tests broke suddenly.
email.sendBulk(generateExpiryEmails(db.getUsers(), Date.now()));
Generally you'd distinguish which function call introduces the error with the function call stack, which would include the location of each function's call-site, so maybe the "low-effort" label is accurate. But I could see a benefit in immediately knowing which functions are "pure" and "impure" in terms of manipulating non-local state. I don't think it changes any runtime behavior whatsoever, really, unless your runtime schedules function calls on an async queue and relies on the order in code for some reason.
My verdict is, "IDK", but worth investigating!
I vaguely remember the problem was one function returned a very structured array dealing with regex matches. But there was something wrong with the regex where once in a blue moon, it returned something odd.
So, the chained functions did not error. It just did something weird.
Whenever weird problems would pop up, it was always passed to me. And when I looked at it, I said, well...
I am going to rewrite this chain into steps and debug each return. Then run through many different scenarios and that was how I figured out the regex was not quite correct.
I don't get how you got there from parent comment.
Pascal just went with a needless syntax split of (side-effectful) methods and (side-effectful) functions.
var users = db.getUsers();
var expiredUsers = getExpiredUsers(users, Date.now());
var expiryEmails = generateExpiryEmails(expiredUsers);
email.bulkSend(expiryEmails);
This is not only much easier to read, it's also easier to follow in a stack trace and it's easier to debug. IMO it's just flat out better unless you're code golfing.
I'd also combine the first two steps by creating a DB query that just gets expired users directly rather than fetching all users and filtering them in memory:
expiredUsers = db.getExpiredUsers(Date.now());
Now I'm probably mostly getting zero or a few users rather than thousands or millions.
This is actually closer to the way the first draft of this article was written. Unfortunately, some readability was lost to make it fit on a single page. 100% agree that a statement like this is harder to reason about and should be broken up into multiple statements or chained to be on multiple lines.
For these reasons one of the things I like to do in Swift is set up a function called ƒ that takes a single closure parameter. This is super minimal because Swift doesn't require parenthesis for the trailing closure. It allows me to do the above inline without cluttering the scope while also not increasing the amount of redirection using discrete function declarations would cause.
The above then just looks like this:
ƒ {
var users = db.getUsers();
var expiredUsers = getExpiredUsers(users, Date.now());
var expiryEmails = generateExpiryEmails(expiredUsers);\
email.bulkSend(expiryEmails);
}The rule I was raised with was: you write the code once and someone in the future (even your future self) reads it 100 times.
You win nothing by having it all smashed together like sardines in a tin. Make it work, make it efficient and make it readable.
What makes it hard to reason about is that your code is one-dimensional, you have functions like `getExpiredUsers` and `generateExpiryEmails` which could be expressed as composition of more general functions. Here is how I would have written it in JavaScript:
const emails = db.getUsers()
.filter(user => user.isExpired(Date.now())) // Some property every user has
.map(generateExpiryEmail); // Maps a single user to a message
email.bulkSend(emails);
In a functional language with pipes it could be written in an arguably even more elegant design:
db.getUsers() |> filter(User.isExpired(Date.now()) |> map(generateExpiryEmail) |> email.bulkSend
email.bulk_send(generate_expiry_email(user) for user in db.get_users() if user.is_expired(Date.now())Question. If you want to do one email for expired users and another for non expired users and another email for users that somehow have a date problem in their data....
Do you just do the const emails =
three different times?
In my coding world it looks a lot like doing a SELECT * ON users WHERE isExpired < Date.now
but in some cases you just grab it all, loop through it all, and do little switches to do different things based on different isExpired.
If you want to do one email for expired users and another for non expired users and another email for users that somehow have a date problem in their data....
But perhaps you can write a more generic function like generateExpiryEmailOrWhatever that understands the user object and contains the logic for what type of email to draft. It might need to output some flag if, for a particular user, there is no need to send an email. Then you could add a filter before the final (send) step.
myCoolSubroutine = do
now <- getCurrentTime
users <- getUsers
forM users (sendEmail now)
sendEmail now user =
if user.expiry <= now
then sendExpiryEmail user
else sendNonExpiryEmail user
If it's just two or three cases I might actually just copy-paste the entire thing. But let's assume we have twenty or so cases. I'll use Python notation because that's what I'm most familiar with. When I write `Callable[[T, U], V]` that means `(T, U) -> V`.
Let's first process one user at a time. We can define an enumeration for all our possible categories of user. Let's call this enumeration `UserCategory`. Then we can define a "categorization function" type which maps a user to its category:
type UserCategorization = Callable[[User], UserCategory]
categorized_users = map(categorize, db.get_users()) # type Iterable[tuple[UserCategory, User]]
type ProcessingSpec = Mapping[UserCategory, Callable[[User], None]
def process_users(how: ProcessingSpec, categorize: UserCategorization) -> None:
categorized_users = map(categorize, db.get_users())
for category, user in categorized_users:
process = how[category]
process(user)
def group_by[T, U](what: Iterable[T], key: Callable[[T], U]) -> Mapping[U, list[T]]:
result = defaultdict(list)
# When we try to look up a key that does not exist defaultdict will create a new
# entry with an empty list under that key
for x in what:
result[key(x)].append(x)
return x
batches = group_by(db.get_users(), categorize)
type BatchProcessingSpec = Mapping[UserCategory, Callable[[Iterable[User]], None]
def process_batched_users(how: BatchProcessingSpec, categorize: UserCategorization) -> None:
batches = group_by(db.get_users(), categorize)
for category, users in batches:
process = how[category]
process(users)
I have been itching to write a functional programming book for Python. I don't mean a "here is how to do FP in Python" book, you don't need that, the documentation of the standard library is good enough. I mean a "learn how to think FP in general, and we are going to use Python because you probably already know it". Python is not a functional language, but it is good enough to teach the principles and there is value in doing things with "one hand tied behind your back". The biggest hurdle in the past to learning FP was that books normally teach FP in a functional language, so now the reader has to learn two completely new things.
expiry_date = DateTime.now!("Etc/UTC")
query =
from u in User,
where:
u.expiry_date > ^expiry_date
and u.expiry_email_sent == false,
select: u
MyAppRepo.all(query)
|> Enum.map(u, &generate_expiry_emails(&1, expiry_date))
|> Email.bulkSend() # Returns {:ok, %User{}} or {:err, _reason}
|> Enum.filter(fn
{:ok, _} -> true
_ -> false
end)
|> Enum.map(fn {:ok, user} ->
User.changeset(user, %{expiry_email_sent: true})
|> Repo.update()
end)
----
Edit: I actually see a few problems with this, too, since Email.bulkSend probably shouldn't know about which user each email is for. I always see a small impedance mismatch with this sort of pipeline, since if we sent the emails individually it would be easy to wrap it in a small function that passes the user through on failure.
If I were going to build a user contacting system like this I would probably want a separate table tracking emails sent, and I think that the email generation could be made pure, the function which actually sends email should probably update a record including a unique email_type id and a date last sent, providing an interface like: `send_email(user_query, email_id, email_template_function)`
What you want is to use a language that has higher-kinded types and monads so that functions can have both effects (even multiple distinct kinds of effects) and return values, but the distinction between the two is clear, and when composing effectful functions you have to be explicit about how they compose. (You can still say "run these three possibly-erroring functions in a pipeline and return either the successful result or an error from whichever one failed", but you have to make a deliberate choice to).
Having a language where "func" defines a pure function and "proc" defines a procedure that can performed arbitrary side effects (as in any imperative language really) would still be really useful, I think
Rust tried that in the early days, the problem is no-one can agree on exactly what side effects make a function non-pure. You pay almost all the costs of a full effect system (and even have to add an extra language keyword) but get only some of the benefits.
_unitOfWork.Begin();
var users = await _usersRepo.Load(u => u.LastLogin <= whateverDate);
users.CheckForExpiry();
_unitOfWork.Commit();
(Clojure) ;; Nested function calls (map double (filter even? '(1 2 3 4)))
;; Using the thread-last macro (->> '(1 2 3 4) (filter even?) ; The list is passed as the last argument (map double)) ; The result of filter is passed as the last argument ;=> (4.0 8.0)
Things like this have been added to python via a library (Pipe) [1] and there is a proposal to add this to JavaScript [2]
1: https://pypi.org/project/pipe/ 2: https://github.com/tc39/proposal-pipeline-operator
Result<Users> userRes = getExpiredUsers(db);
if(isError(userRes)) {
return userRes.error;
}
/* This probably wouldn't actually need to return a Result IRL */
Result<Email> emailRes = generateExpireyEmails(userRes.value);
if(isError(emailRes)) {
return emailRes.error;
}
Result<SendResult> sendRes = sendEmails(emailRes.value);
if(isError(sendRes)) {
return sendRes.error;
}
return sendRes; // successful value, or just return a Unit type.
Result<SendResult> result = pipe(getExpiredUsers(db))
.then(generateExpireyEmails)
.then(sendEmails)
.result();
if(isError(result)) {
return result.error;
}
Optional.ofNullable(getExpiredUsers(db))
.map(EmailService::generateExpireyEmails)
.map(EmailService::sendEmails)
.orElse(null);
C++ also added a std::expected type in C++23:
auto result = some_expected()
.and_then(another_expected)
.and_then(third_expected)
.transform(/* ... some function here, I'm not familiar with the syntax*/);We should never be too extreme on anything, otherwise it would turn good into bad.
No one _should_ do that, but that's a common enough problem (that usually doesn't get found until code is running in production). I suspect with the rise of vibe coding, it's going to happen more and more.
Still smells like in such a case the developer avoids the complications of abstraction or OOP by making the user deal with it. That's bad API design due to putting ideology before practicality or ergonomics.
The query shape would be:
active_users = Query(User).filter(active=True)
def active_users_except(exclude):
return active_users.filter(User.id.not_in(exclude)
Then at execution time, the objects representing query expressions are rendered into queries and sent to the database:
exclude_criterion = rude_users() # A subquery expression
polite_active_users = load_records(
active_users_except(exclude_criterion)
)
@dataclass
class ActiveUsers(QueryTemplate):
active_if: Expression = User.active == true()
@classmethod
excluding(cls, bad_set):
return cls(
and_(
User.active == true(),
User.id.not_in(bad_set)
)
)
@property
def query(self):
return Query(User).filter(self.active_if)
load_records(
ActiveUsers.excluding(select_alice | select_bob).query
) IEnumerable<User> getExpiredUsers(DbSet<User> users)
=> users.Where(u => u.ExpiresAt < DateTime.UtcNow);
I'm building a new project, Typegres, on this same philosophy for the modern web stack (TypeScript/PostgreSQL).
We can take your example a step further and blur the lines between database columns and computed business logic, building the "functional core" right in the model:
// This method compiles directly to a SQL expression
class User extends db.User {
isExpired() {
return this.expiresAt.lt(now());
}
}
const expired = await User.where((u) => u.isExpired());
(Also, real-life systems of course do things inefficiently all the time)
In what application would you load all users into memory from database and then filter them with TypeScript functions? And that is the problem with the otherwise sound idea "Functional core, imperative shell". The shell penetrates the core.
Maybe some filters don't match the way database is laid out, what if you have a lot of users, how do you deal with email batching and error handing?
So you have to write the functional core with the side effect context in mind, for example using query builder or DSL that matches the database conventions. Then weave it with the intricacies of your email sender logic, maybe you want iterator over the right size batches of emails to send at once, can it send multiple batches in parallel?
You’d be surprised! I have worked on a legacy PHP service which did something very similar
Generally, performance is a top cause of abstraction leaks and the emergence of less-than-beautiful code. On an infinitely powerful machine it would be easy and advisable to program using neat abstracrions, using purely "the language of" the business. Our machines are not infinitely powerful, and that is especially evident when larger data sets are involved. That's where, to achieve useful performance, you have to increasingly speak "the language of" the machine. This is inevitable, and the big part of the programmer's skill is to be able to speak both "languages", to know when to speak which one, and produce readable code regardless.
Database programming is a prime example. There's a reason, for example, why ORMs are very messy and constitute such excellent footguns: they try to gap this bridge, but inevitably fail in important ways. And having and ORM in the example would, most likely, violate the "functional core" principle from the article.
So it looks like the author accidentally presented a very good counterexample to their own idea. I like the idea though, and I would love to know how to resolve the issue.
email.bulkSend(generateReminderEmails(getExpiredUsers(db.getUsers(), fiveDaysFromNow)));
If you pick and recommend a pattern where filtering should happen separately from retrieving, all your code will be bad
Give a man a fish / teach a man to fish, but bad.
Replace it with `getUsers(filters)` or even a specialised function, and it starts making more sense.
It's exactly this - I do regret using "db" a bit now after reading all of the comments here, as it's taken away focus from the main point. But yes, the post had to fit on a single page, and I needed to pick something that most engineers would be familiar with.
But even if not, the example from the article is just hypothetical. `db.getUsers()` could be something that just retrieves rather efficient `[UserEmail, ExpiryTime]` pairs, and then you'd have to have a pretty enormous user base for it to not scale (a couple of million string/date pairs in memory should be no problem).
I fixed a performance issue at some point where a missing index meant a scan of millions of rows every login. It worked, and could log in 3 people per second or so. It was still terrible code.
Of course by "invented" I mean that far smarter people than me probably invented it far earlier, kinda like how I "invented" intrusive linked lists in my mid-teens to manage the set of sprites for a game. The idea came from my head as the most natural solution to the problem. But it did happen well before the programming blogosphere started making the pattern popular.
It is not, it is being very specific about what it means and what it is referring to
We have tens of thousands of lines of code for the platform and millions of workflow runs through them with no production errors coming from the core agent runtime which manages workflow state, variables, rehydration (suspend + resume). All of the errors and fragility are at the imperative shell (usually integrations).
Some of the examples in this thread I think get it wrong.
db.getUsers() |> filter(User.isExpired(Date.now()) |> map(generateExpiryEmail) |> email.bulkSend
What you really want is (in TS, as an example):
bulkSend(
userFn: () => user[],
filterFn: (user: User) => bool,
expiryEmailProducerFn: (user: User) => Email,
senderFn: (email: Email) => string
)
I chose this approach because writing integration tests with LLM calls would make the testing run too slowly (and costly!) so most of the interaction with the LLM is simply a function passed into our core where there's a lot of logic of parsing and moving variables and state around. You can see here that you no longer need mocks and no longer need to spy on calls because in the unit test, you can pass in whatever function you need and you can simply observe if the function was called correctly without a spy.
It is easier than most folks think to adopt -- even in imperative languages -- by simply getting comfortable working with functions at the interfaces of your core API. Wherever you have I/O or a parameter that would be obtained from I/O (database call), replace it with a function that returns the data instead. Now you can write a pure unit test by just passing in a function in the test.
I am very surprised how many of the devs on the team never write code that passes a function down.
Even Python examples in trainings that look functional might not be. They put the function calls in as arguments. The beginner thinks the function returns some data, that would be in a variable, and they are implicitly passing that variable. Might as well, for readability, do the function call first to pass a well-named variable instead.
That was my experience. That plus minimizing side effects in functions. I've yet to really learn functional programming where I'd think to pass a function in an API. What are the best articles or books for us to learn that in general or in Python?
The book Architecture Patterns in Python by Percival and Gregory is one of the few books that talks about this stuff using Python. It's available online and been posted on HN a few times before.
You write a concrete set of steps, but delegate the execution of the steps to the caller by invoking the supplied functions at the desired time with the desired arguments.
The difference between the two is that when you inject an object, the receiving side must know a potentially large surface area of the object's behavior.
When injecting a function, now the receiving side only needs to know the inputs and outputs of the singular function.
This subtlety is what makes this approach more powerful.
What if a FCF (functional core function) calls another FCF which calls another FCF? Or do we do we rule out such calls?
Object Orientation is only a skin-deep thing and it boils down to functions with call stack. The functions, in turn, boil down to a sequenced list of statements with IF and GOTO here and there. All that boils boils down to machine instructions.
So, at function level, it's all a tree of calls all the way down. Not just two layers of crust and core.
You’ll find usually that side effect in imperative actions is usually tied to the dependencies (database, storage, ui, network connections). It can be quite easy to isolate those dependencies then.
It’s ok to have several layers of core. But usually, it’s quite easy to have the actual dependency tree with interfaces and have the implementation as leaves for each node. But the actual benefits is very easy testing and validation. Also fast feedback due to only unit tests is needed for your business logic.
I also see that lately "code quality" is the least concern of most (even software product) companies, just ask AI to write code in a single file / module / class - then launch feature and fix if you have to. I could see that in a few years things will be extremely messy (but who can say).
There's a link with more info at the top. I'm not sure why this one in particular made it to the front page of HN.
Have to ship it non matter what.
Hopefully by 2045 these ideas will have gotten a little more traction.
Or is it that the example in the article is a bit poor?
> But personally, I don't use tell-dont-ask. I do look to co-locate data and behavior, which often leads to similar results. One thing I find troubling about tell-dont-ask is that I've seen it encourage people to become GetterEradicators, seeking to get rid of all query methods.
I mean, what if you want to do IO and have mutable data structures inside a do block? I'm afraid I'm going to have to prescribe you a monad transformer. Be careful of the side effects.
[0] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
[1] https://en.wikipedia.org/wiki/Hexagonal_architecture_(softwa...
[0]: https://www.destroyallsoftware.com/talks/boundaries
[1]: https://www.destroyallsoftware.com/screencasts/catalog/funct...
Hex is kind of a PITA for ground up projects, but if you are doing something where you know multi-platform/cloud/device whatever is important it is cool.
Some things are flat out imperative in nature. Open/close/acquire/release all come to mind. Yes, the RAI pattern is nice. But it seems to imply the opposite? Functional shell over an imperative core. Indeed, the general idea of imperative assembly comes to mind as the ultimate "core" for most software.
Edit: I certainly think having some sort of affordance in place to indicate if you are in different sections is nice.
It can be done "functionally" but doesn't necessarily have to be done in an FP paradigm to use this pattern.
There are other strategies to push resource handling to the edges of the program: pools, allocators, etc.
Consider your basic point of sale terminal. They get a payment token from your provider using the chip, but they don't resolve the transaction with your card/chip still inserted. I don't know any monad trick that would let that general flow appear in a static piece of the code?
Can you implement it using functional code? Yes. Just make sure you wind up with partial states. And often times you are best off explicitly not using the RAI pattern for some of these. (I have rarely seen examples where they deal with this. Creating and reconciling transactions often have to be separate pieces of code. And the reconcile code cannot, necessarily, fallback to create a transaction if they get a "not found" fault.)
Yes, the monadic part is the functional core, and the runtime is the imperative shell.
> Consider your basic point of sale terminal. They get a payment token from your provider using the chip, but they don't resolve the transaction with your card/chip still inserted. I don't know any monad trick that would let that general flow appear in a static piece of the code?
What do you mean by Monad trick? That's precisely the kind of thing the IO monad exists for. If you need to fetch things on an API: IO. If you need to read/save things on a DB: IO. DB Transaction: IO.
Granted, in trying to find some examples that stick in my memory, I can't really find any complete examples anymore. Mayhap I'm imagining a bad one? (Very possible.)
If the transaction object is serializable you can just store it in a DB, for example. If it's some C++ pointer from some 3rd-party library that you can't really serialize and gotta keep open, you gotta keep it in memory and manage its lifetime explicitly, be it a REST web server, in Haskell or in a C++ app.
That's not what functional core, imperative shell means though. It's a given that CPUs aren't functional. The advice is for people programming in languages that have expressions - ruby, in the case of the original talk. The functional paradigm mostly assumes automatic memory management.
I'm sympathetic to the idea, as you can see it in most instruction manuals that people are likely to consume. The vast majority of which (all of them?) are imperative in nature. There is something about the "for the humans" layer being imperative. Step by step, if you will.
I don't know that it fully works, though. I do think you are well served being consistent in how you layer something. Where all code at a given layer should probably stick to the same styles. But knowing which should be the outer and which the inner? I'm not clear that we have to pick, here. Feel free to have more than two layers. :D
Indeed. It's all well and good to impart some kind of flavour into your code and call it functional, but transactions do not give a crap about style.
A transaction needs to be able to 'back out' to fulfill 'all-or-nothing' semantics. Side effects are what make this impossible.
And that's what this thread is filled with, and that's what I'm pushing back against.
> the RAI pattern is nice
> indicate if you are in different sections is nice
Style doesn't matter, flavour doesn't matter, wants don't matter, code "quality" (whatever that means) doesn't matter, niceness doesn't matter.
A transaction can be rolled back. If it can't, it's not a transaction.
To that end, any style that tries to move those two time periods closer together in code is almost doomed to have some either hard to reason about code, or some tough edge cases that are hard to specify.
(Granted, I'll note that most transactions that people are dealing with on a regular basis probably do open and close rather close to each other.)
I don't know what you mean by going a little further. I said transactions are 'all or nothing', not 'all or nothing or 99%'.
I grant that for things that are purely informational, this is not necessarily as accurate. But as the things reasoned about in a program get larger and larger, transactions span longer and longer timelines. With "all or nothing" not being nearly as clear cut as it is in smaller examples.
My go to examples are things like vending machines. (Granted, that almost certainly just shows my bias for state machines all the way down.)
I always feel like I have to "maintain" code so I usually get bored after 3k lines of code, but truth is code doesn't have to be maintained if we like it the way it is, which obviously includes all the functionality that comes with it.
I mean it's not much, but the concept just resonates with me and I want to share it. Sad I can't share even simple opinion nowadays ...