Hacker News Reader

-- Functional programming does not work well in programs which handle a lot of states

Apache Spark is written in Scala. It can handle extremely large amounts of state across large clusters of machines. Is the argument here that Scala is not purely functional? Or what am I missing?

View on HN

By "lots of state" I don't think he meant "lots of data", but rather data that get changed a lot: Q&A forum, CRUD application, etc.

Functional paradigms can still be used (e.g. HackerNews, though let's be honest...I love it but it's not the most challenging feature set). But FP isn't as natural of a fit as the problems chosen to introduce it.

Not saying your wrong about what he meant, but Spark can also keep state across the Cluster, not just data. Check out the "UpdateStateByKey Operation" in the following page.

http://spark.apache.org/docs/latest/streaming-programming-gu...

I haven't taken a look at the Spark codebase, but it is very easy to write Scala code that is not purely functional.

Reply on news.ycombinator.com