Is Haskell more academic in nature or used heavily in Production environments?
Is there an application sweet spot/domain (say Artificial Intelligence/Machine Learning, etc) where it shines over using other languages (I am not talking about software/language architectural issues like type systems or such)?
I have no experience with Haskell but do use functional languages such as Erlang/Elixir on and off.
State of the Haskell ecosystem: https://github.com/Gabriel439/post-rfc/blob/master/sotu.md
The topics are roughly sorted from greatest strengths to greatest weaknesses.
Each programming area will also be summarized by a single rating of either:
Best in class: the best experience in any language
Mature: suitable for most programmers
Immature: only acceptable for early-adopters
Bad: pretty unusableTake out the programs that are for writing code, ghc, shellcheck, darcs etc. And what are you actually left with? git-annexe, pandoc, xmonad is hledger worth knowing about if you don't care what language it was written in? (Maybe it's amazing, first I've heard of it).
Whenever I bring this up, people find it upsetting. The Haskell community might need to get past both of those things. 1.0 was 27 years ago. So many fabulous programmers have gone down the road for entertainment or enlightenment so where are your damn apps if you're a language that doesn't suck for writing them, no really, where are they?
For example, I think many would struggle to name many open source user application written in Clojure as well.
However, the work also includes a runtime platform that is more low-level, including building our own container system, talking to the Linux kernel, using cgroups/resource management, and distributed message passing - areas where languages such as Go have found a niche, and may be classified as Real World. However for us, Haskell's type-safety, runtime performance, and extensive ecosystem has been a boon even in this domain. We effectively use it as an (incredibly powerful) general-purpose language here, and it's worked more than fine.
We're currently at around 15,000 lines of code with a team of 5 Haskellers, and it hasn't really been a problem regarding performance, understanding the codebase, or with newcomers to the team.
(plug - we're at https://www.github.com/nstack/nstack and are hiring more Haskellers)
http://programatica.cs.pdx.edu/House/
Also, COGENT for lowest-level stuff being wrapped for use in Haskell somehow might be interesting. Used on ext2 filesystem already.
I haven't seen COGENT before - will take a look over the weekend - thanks!
For the massively parallel workloads you find in data science, it seems like you'd benefit a lot from the wealth of container orchestration tools around Docker (swarm, Rancher/Cattle, Kubernetes) in order to easily scale out your functions. Especially when many companies already have these set up for their more vanilla applications.
This is an example I've seen that can leverage a docker swarm for invoking functions, loosely modeled after AWS Lambda: https://github.com/alexellis/faas
We're really lucky also to have one of the main `rkt` developers joining us soon to work on the container side.
Weirdly, what Haskell is good at is generality. Which is much the same as for LISP.
I'd say the Haskell community remains academic/hobbyist, but that's not to say there aren't plenty of people doing Real Work in it.
It is a very good language once the type system makes sense and you can reliably reason about the lazy nature of it.
For instance, if you were to write a CAD program, you probable want to be spending your time thinking about algebra and geometry, and not about little details like how an array is laid out in memory or when memory needs to be freed. The latter might be important for optimizing the last bit of performance out of a tight loop, but a lot of problems are hard enough on their own that just coming up with any solution that works is enough trouble on its own.
It's also good for problems where you want pretty good performance but don't have to be as fast as C, or where you want to take advantage of multi-core parallelism but don't want to deal with the usual complexities of threaded code.
Persistent data structures are really nice for certain kinds of applications, such as web services, where you have some complex internal state that needs to be updated periodically. Error handling is really easy when you can just bail out when something fails, since none of the state was ever modified-in-place and your old pointer to the old state is still valid.
The powerful static type checker makes Haskell a good fit for applications where it's really important that you calculate the right answer, or where you don't want to spend much time tracking down weird bugs.
Haskell isn't so good at applications where you need hard real-time guarantees or you want to have very fine control over the memory layout of your data structures or you want to be able to easily predict CPU or memory usage.
I'd say that people mostly do not know what is running in industry. It could be any share of anything. We get a hint looking at job offers; Haskell is small there, but not unheard of.
On the sweet spot, Haskell is great for modeling complex domains; it's great for long-lived mostly maintenance stuff, since refactoring is so easy; it tends to generate very reliable and reasonably fast (faster than Java/.Net) executables, but hinders complete correctness proofs and has some grave speed issues that can ruin your day if you hit them.
I would like to know how is it to manage a team of Haskell developers. I do expect it to be better than Java/.Net in forcing safe interfaces between developers (thus making it reasonably easy to survive a bad developer in a team), but I never saw it in practice.
well if anyone does know what's out there then HN crowd would be a pretty good suspect in that regard.
I personally have done my fair share of consulting and prof services engagements - dozens of them across all kinds of industries - I've never seen a Haskell shop.
Java, Python, JS, Golang, Scala, Clojure, .Net, C, PHP - definitely out there in the field. Haskell - not so much.
HN is a comically bad barometer for anything other than what's trendy in norcal.
Compare with how many Java consultancies do you think you can find on a quick search?
I don't think the Haskell development labor market is nearly liquid enough to rely on consultants. For full time jobs the picture is very different.
Then he'll have to look up some that he can't name off the top of his head? I can't name any Java consultancies, but I'm nonetheless confident that there are plenty...
With regard to Haskell, there are a lot of people interested in Haskell work compared to the number of positions presently available.
I'd argue Python is much more prevalent with researchers than Haskell.
As for particular places where it shines, I don't know of any in particular. I remember when taking my Declarative Programming course in university my lecturer loved to talk about an experiment the military did where they challenged a few teams to implement some kind of radar system in their chosen language and evaluated them based on time taken, correctness and code complexity (LOC) at the end. The Haskell team performed very very well. I can't find any details on this experiment though...
Probably this one: http://www.cs.yale.edu/publications/techreports/tr1049.pdf
But take it with some sacks of salt -- it's a single research paper. It should be replicated, peer reviewed, etc to have any merit. It's methodology could be totally crap for example...
I love it for general back end web development. I can actually solve problems so they stay solved.
How did you convince the client, and why did you choose Haskell for this over, say, .NET, which throws a web service up quicker than it took me to type this? What sort of productivity are you achieving?
At SQream (www.sqream.com), Haskell is used for CLI, SQL parser, SQL language compiler, SQL optimizations and a variety of tools.
We also use Haskell to generate Python code for testing (think: describing a testing scenario and sets of queries, translating into a runnable, predictable Python script that's easy to debug)
Haskell's strength at expressing and transforming AST's, in pure functional, understandable, testable code seems to be the key here.
We also employ QuickCheck rather extensively, which makes testing millions of scenarios really straightforward.
That's what drove me to languages like OCaml and Rust, which try to solve this tradeoff in a different way: Same thing about strictness and correctnes (despite slight differences in the type system), but lazy evaluation is only provided when explicitly asked for.
The cool thing about OCaml is that you can actually reason about performance, as long as you ignore memory issues (memory usage, garbage collector runs, etc.). Rust is heavily inspired by OCaml and allows you to also reason about correctness and performance of memory usage.
If you ignore mem issues you can also reason about the performance of Haskell :)
If we want to reason about perf, I think Rust is currently leading in terms of a modern language that allows perf reasoning.
Really? My guess (based on close to complete ignorance) would be that laziness would make reasoning about performance hard. Can you ELI5 why my guess is wrong?
Because of it potentially hogging memory! Now we were told to ignore mem issues by the commenter. I just point out the contradiction.
But all of that is just because data stays in memory longer than expected.
If you ignore that, laziness has a small performance impact, but it's less than the performance impact of interpreting code, which is something many people consider completely acceptable.
That may not be "a performance issue caused by laziness", in your terms, because the laziness isn't causing performance issues. But it's laziness making a performance issue hard to find.
Does my scenario happen? Is it common? Is it easy to analyze/find/fix when it does happen?
Most generally, does laziness make it harder to reason about performance?
The 'memory stays around indefinitely' problem is caused by not cleaning up references to generators. You can make your own xrange() clone in Python easily, and start consuming values, but if you stop before consuming all of them (assuming it's not an infinite generator), that xrange() generator object still lives and has to keep around whatever references it needs to generate the next value if you ask it for one. When your whole language is based around laziness, you might have a lot of these and it may not be clear what references are still around that keep the GC from cleaning them up.
I wouldn't say reasoning about performance is necessarily more difficult, and profiling will light up a slow part of a lazy pipeline just as well as an eager one. My own struggles with laziness in Clojure were all around my own confusions about "I thought this sequence was going to be evaluated, but it wasn't!" which may have led to unnecessary calls to (doall), which forcibly walks a sequence and puts the whole thing in memory.
Profiling is a bit of a black art in any case. I've seen horrible performance problems in python when we added enough code to cause cache thrashing to a loop. No new code was slow, but the size of it crossed a threshold and things got suddenly bad.
Some performance problems in Haskell are hard to track down, but most are pretty easy to find and fix. It's basically the same as every other language in that way.
Good Type safety seems to be an acquired taste.
Somehow reminds me of Structs/Records in Erlang/Elixir and using pattern matching, conditions and guard clauses to enforce validity or logic.
I think it's just a pretty good general purpose language, for the most part? I know that's not very exciting of an answer, but I do consulting for people and I've seen Haskell for everything our clients do, from web development (a lot of web development), compiler development, distributed OLAP style analytic processing on large data volumes, custom languages/DSLs, machine learning, hardware/FPGA design work, desktop software (flight tracking/schedule software, used by on-ground airport personnel, with some incredibly beautiful code), cryptocurrency and cryptography etc etc. There are the occasional research-y things, and we've also just done things like that for random institutions or companies (e.g. prototyping RINA-style non-TCP network designs.)
I wouldn't say any of these were a particular "sweet spot". Though some things like compiler development or web development are pretty pleasurable. The language and libraries make those pretty good. Web development is one of the more popular things now especially. Everyone has to have their thing in a browser now, and there's a fair amount of library code for this stuff now. Definitely one of the stronger things.
You do get the occasional problem that just has this perfectly elegant solution at the core and that's always an awesome bonus. One of my favorites was the aforementioned flight software. It had this very strict set of bureaucratic guidelines but all the core logic ended up having a very nice, pure formulation that ended up being about 8 functions used in a single function-composition pipeline. (We use this sometimes as an example in our training of real solutions for real problems.) A lot of software is much more boring and traditional.
Mostly, it works well and stays out of my way, and lets me solve problems for good. And I really like the language in general compared to most other languages; it hits a very unique sweet spot that most others don't try to touch.
I do admit the sort of high level, abstract shenanigans are fun and interesting too, but I think of it more as a fun side thing. Just this week I literally wrote the documentation for a data type, describing it as an "Arbitrary-rank Hypercuboid, parameterized over its dimension." Luckily I do not think you will ever have to use that code in reality, so you can breath easy. :)
---
EDIT: and to be fair, there are definitely some problems. Vis-a-vis my joke, we could all collectively use better "soft" documentation. I think some problems like "operator overload extravaganza" are overblown due to unfamiliarity. Most real codebases aren't operator soup; but bigger problems are things like "When to use the right abstraction" are real problems we have hard times explaining.
I don't think performance is much harder than most other languages, but we definitely don't have tools that are as nice in some ways, nor tools that are as diverse. The compiler is also slow compared to e.g. OCaml. Some parts of the ecosystem are a lot worse than others; e.g. easy-to-use native GUI bindings are still a bit open. Windows is a bit less nice to use than Unixen.
There's a big gap between training in-person vs online/self taught content too, IME.