Why hasn't functional programming taken over yet?

What do you mean by a stack of state?

If you use the ST monad instead of State, you get actual mutable variables. STArray lets you write typical imperative algorithms that work with mutable arrays, like sieve of Eratosthenes or dynamic programming.

Imperative sieve of Eratosthenes:

https://gist.github.com/rik0/1171446#file-erat1-hs

I meant "hidden" state because functional programs still reason about state, but more explicitly. In a functional style you can't just call a function that relies on hidden state somewhere, you have to pass that state in!

But with partial application and currying you could pass the state in at any point in the program, so all the functions on the inside are pure, while state exists on the edges.

20? More like 40. Until 2005 or so few considered this "pure" style essential to FP.

It depends on what kind of performance problems you have.

Some algorithms are hard to express in pure, strict functional settings. (There's even an impossibility theorem for a few algorithms, but that theorem doesn't apply to pure, lazy languages.) That being said, the worst asymptotic slowdown is logarithmic---because you can always simulate random access memory.

On a more practical side, you have constant factors to worry about, and things like caches. Look at eg Don Stewarts work for how to deal with these in a functional language.

(And, in practice, there's no shame in throwing in the towel and either interfacing with other languages, or even switching to them.)

Sure, but it isn't being modified by the expressions in the cells.

Either that or take a look at Elixir. All the 9 9's reliability and distributability of Erlang with a more Ruby-like syntax.

Sorting and filtration are super bread-and-butter FP. The "hard" bits are the IO to get the results and to print them out. Solve the first with a nice HTTP client lib like Wreq, bring in the IO language to do it, finish it off using putStrLn to print. It's neither the same mechanism as you're used to not difficult.

> The last time I looked at Haskell, it needed ~700MB of dependencies before I could even get started.

ghc is big, indeed. There are smaller alternate compilers and interpreters (that no one uses). Compiled executables are usually smaller than ghc, which is probably the biggest Haskell programs ever written.

There are ways to write things that are similar with Haskell. Just like there are design patterns for functional techniques in imperative languages, there are design patterns for imperative techniques in functional language....

So for example, here you could make a nice little EC2 DSL that would get your stuff (for the actual Haskellers: Free Monad):

    spotPrices :: Region -> EC2 [SpotPrice]
    spotPrices = GetSpotPrice -- DSL Leaf, executed by runEC2

    spotPricesForAllRegions :: [Region] -> EC2 [SpotPrice]
    spotPricesForRegions regions =
        concat <$> MapM spotPrices regions
    

Your spot price update function becomes something even simpler now:

    runEC2 :: EC2 a -> EC2Config -> IO a
    runEC2 = stuff -- this is where you call whatever EC2
    -- lib is available

 
    updateSpotPriceFile :: [Region] -> IO [SpotPrice]
    updateSpotPriceFile regions = do
        spotPrices <- runEC2 (spotPricesForAllRegions regions)
                             (myEC2Configuration)
        writeToFile (toJSON spotPrices) "spotprices.json"
        return spotPrices

I'm not going to port all of it, but with a more functional version you could avoid having to keep on making EC2 clients all over the place, and you could keep your config in one place.

The problem is that it's pretty obtuse if you don't know the right design patterns. There's a myth in FP circles that design patterns are for OOP only ("we have functions, we don't need observer patterns!"), but it's a myth. But if you get good, you can write some pretty concise code + get all the goodies.

Haskell has the whole GHC thing, and also the issue of lack of good library documentation (or lack of libraries on certain stuff)...

Something like this in Scala: it doesn't have the json read/writing but only because I lost interest, not because it's intrinsically hard.

https://gist.github.com/hythloday/6246a352c9e9d0150a4b

If I get some extra time I'd love to post a build able Haskell clone of this.

No, you may have to think about it a little more, especially when first getting acquainted with recursion, but the recursive solution is likely to be less complex and more robust. It's the repetition of a simpler idea, and so there are fewer ways to screw-up the implementation.

Why? First, loops are only a special case of recursion, and trying to shoehorn everything into that special is actually hard to read.

Second, functional programming usually emphasises combinators like map, filter, reduce/foldr. Those are easier to read than either naked recursion or loops.

Recursion is great! I find recursive algorithms much easier to understand

Explicit use of recursion in functional programming is widely considered to be low level and bad form, like writing all your loops with gotos in a procedural/OOP language (as was the primary argument of the Bananas, Lenses, Envelopes, and Barbed Wire paper), it's just commonly used in intro material in a "this is how you would do this by hand" fashion. For things that fall outside of that, you always have the option of monadic loops if it makes the code clearer.