Are Databases and Functional Programming at odds?
stackoverflow.com
stackoverflow.com
The relational database and your imperative program are two different things, trying to "unify" them will not work.
There's a certain incompatibility that prevents them being truly powerful without bending your schema to the will of the ORM, or without sacrificing a lot of the potential functionality SQL has to offer (and thus the readability of your code as you try to shoehorn these things in).
I suppose NoSQL solutions help fill that niche, though.
The real get-to-the-root-problem answer, which I don't see on StackOverflow right now, is to come to a better understanding of what functional programmers mean by statelessness. Broadly: A function ought to be a function. That is, when called with the same parameters, it should return the same result. A functional programmer says that in an imperative program, there is an implicit additional parameter passed to every function, call it World, which contains vast quantities of state, global variables, global references, the IO state of the world, the ability to open sockets, etc., and that every function implicitly returns a new World value with unbounded changes made to it. Functional programming removes this implicit passing of the World value, and replaces it with an explicit version. (Or rather, the strong, modern version of "Functional Programming", which is implicitly what was specified when talking about the importance of statelessness.) In Haskell, this is the IO datatype used as a monad.
A stateless function is one that does not examine World and depends solely on its non-World parameters. You can write them in non-functional languages, they just don't particularly provide you any assistance. Functional languages are ones that do provide you assistance, such as the State monadic value, which contrary to popular belief is in fact fully stateless when the arguments to all the relevant functions are de-sugared and examined. (It has a state-like effect as the programmer thinks about it, but is actually just syntax sugar for threading a value through a series of functions, as older-school functional languages like Erlang do.)
Functional languages conceptually allow functions to apply to the real world by threading a World value through some functions, and ensuring that World values can't be reused. (How they literally function may differ in ways you don't care about for now.) This is what the IO monad is doing in Haskell, ensuring there's no way to reuse a given World psuedo-value.
So, no, functional languages have no problems talking to the database when passed the relevant World value. (Hypothetically they could be passed a restricted value that would let them only talk to the database, rather than have full access to all IO.) You just won't be able to crack open a connection to the database and start talking to it unless you're in a context where that is permitted. You won't be allowed to do it without realizing it, declaring it, and explicitly controlling it.
(Also, as is the way with this sort of post, there's some ways to quibble with it. I was trying to keep it not Haskell specific, but that sort of makes it hard to maintain consistent terminology, especially when speaking to novices. Perhaps I should just have gone for Haskell only.)
I think that maybe for "industrial" programmers, we need a language, perhaps something like Scala, where the default is pure functions and immutable data, but you can tag things (functions & data) as "mutants", er, "mutable"/"mutates", and those tags "taint" what those elements can be combined with. Call something with side effects from something that should be pure, and get an error, that sort of thing. Then, teach people to use mutable data as sparingly as possible. Maybe wrap database updates in some kind of (small!) message handling actor?
If you enforce that, you get Haskell, where you have no choice but to wrap your head around isolating your IO layer from everything else. If you don't enforce that, you don't get very much material difference in practice.
I'm concerned that in practice, 99% purity isn't very useful. We shall see. (You get advantage from 0 - 50%, then I think a long plateau, then a sharp spike right at 100%. It can be hard to make the leap, since there's no way to ease into it.)
"The entire "monad" discussion is pretty hard to get your head around. Perhaps part of the problem is that it represents not just side effects, but also forced ordering of operations (albeit, largely to allow for those effect-full interactions with the outside world)."
Well, part of the problem is that "monad" doesn't mean any of those things. It's an interface (very much like the Java sense of the term) which has some implementations that enforce those properties, and other implementations that do not. Which is to say that yes, it is hard to get your mind wrapped around, but this is largely because of the spray of terrible, terrible explanations of the monad interface sprayed about by people who barely understand it themselves.
The best way to actually learn them is Learn You a Haskell; if you've already learned basic Haskell syntax you can leap straight to chapter 11[1]. (Do not jump straight to the Monad chapter 12. Start at 11. Start earlier if you don't know the syntax.)
"Monad" is an adjective, not a noun.
[1]: http://learnyouahaskell.com/functors-applicative-functors-an...
I did do at least some LISP back in the 80s in college, although it was limited to puzzles and mini expert system type problems (enough to see that pure functions were a good idea), but not to do what most people would think of as "applications" with any kind of data feed connectivity. By the late 80s, AI seemed to be moving to neural net ideas, so we didn't dwell on LISP much in the other AI related class I took.
The paying the bills in the real world forces you into a plethora of Turing tar pit languages :-)
OTOH, simply trying to explain the idea of "pure functions" to one of my Java programming coworkers a few years ago was stunning. Me: look, you can easily understand the workflow this way -- the routine is short, and all the data used is in the parameter list and the result returned. Him: hmm, I guess that might help with debugging.
It's a great idea for a new paradigm of functional relational programming.
I think it has some excellent insights, all developers should read it.
One principle for these sorts of applications is the "command/query separation". In essence, a procedure exposed at an interface level should either (a) do something, and do exactly what the user expects, or (b) answer a question about the system without computational effects. The case of (a) might be called "side effects", but that's a misnomer in this case because the requested state changes are the main effects.
Functional programming is really about having the most basic "building block" of a program be a referentially-transparent function, and to handle stateful effects either using an abstraction such as an Action DSL (which represents a collection of stateful behaviors, which are computed statelessly and then executed or "committed" in a stateful environment, relegating the statefulness to one "commit" function) or a monadic representation. In contrast, imperative programming uses computational actions as the atomic "building block".
For small programs, either imperative or functional programming will work. Imperative programming isn't evil; sometimes (but not often) it's the right approach. The problem emerges as programs get larger and more complicated, and more hands pass over the code. If a function is referentially-transparent, its semantics are usually tractable. If proper tests exist for it, optimizing the function (e.g. for performance) shouldn't affect the rest of the system. The constraint to which the function's maintainer is held is that the semantics shouldn't change, and any alteration that does change its semantics is an objective bug. The function might be memoized (that's a private "side effect" that doesn't affect referential transparency) but it still has to behave the same way. That's a really good property. In an imperative language where functions are replaced by computational actions, these actions are frequently made more complex and featureful as time goes on, and unexpected interactions emerge. Semantics can be defined for imperative programs, but doing so is more complicated and it's rarely done in real-world professional programming.
Most clusterfuck awful systems in software, for the record, are complex systems with imprecise semantics. I would say that 90% of software failures I've seen can be characterized in this way. This sort of thing can happen in any language, but it seems to be a lot more common in imperative programming, especially in verbose languages like Java or C++. It's just much harder to enforce precise semantics over an anything-goes imperative style of programming than a functional one.
As for databases... well, reality is that purely functional programming isn't always possible. We're not just computing mathematical functions; we're trying to do things in the physical world. We need persistence, and that means telling a robotic arm to write bytes on magnetic media (stateful effect). There are several approaches that one can use. One that I like (and it has a lot of desirable properties in distributed systems) is the append-only approach: update and delete operations simply aren't allowed, although new records can shadow old ones in the context of time-sensitive queries. For example, the temperature in New York is 56 degrees now (8:30 am, May 7). In six hours, it might be 70. That doesn't mean we throw the old record away. It's still valid: the temperature was 56 at 8:30. The answer to the query, "What was the temperature at 8:30 in New York?" should never change. On the other hand, the answer to the query, "What is the most recent temperature in New York?" will change all the time.
I often want to promote this way to coworker, any reference on this topic to help me convince them?