Dispatch Author Responds to the Scala Email | Hacker News Reader

Dispatch Author Responds to the Scala Email

code.technically.us

61 pointsmcfunley10 comments

I think it is fair to criticize Dispatch in the following sense: "is this a library you should use in production code?". I think the answer is no, because it has an API that, while terse, is difficult to read and maintain; and furthermore it pays very little attention to robustness performance concerns (like failure accrual and more sophisticated connection pooling) that quickly become important in production software.

Dispatch is a good example of a library that is more fun/pretty than practical for systems engineering -- which is not uncommon in the Scala community, as Coda rightly points out. It's not bad to be fun/pretty rather than practical, but it is right to criticize that (strongly) from an engineering perspective.

I'm biased, but: Finagle has a substantially more robust HTTP client and server library than Dispatch, and it has been stressed at far larger deployments than Dispatch. If you're writing a high-performance system that needs to make tens of thousands of HTTP requests per second, Finagle is a good choice and Dispatch is not.

Finagle is a great library. We were able to pretty quickly write a HTTP load-testing tool; features like failover and retries give the impression that it's a battle-tested library.

However, like all the Twitter Scala libraries, it's currently stuck at 2.8.1 (2.9.0, the next version, was released May 2001). Most new Scala libraries are only being released for 2.9.x, meaning that most developers are forced to make a choice between using Twitter's libraries or other, newer ones.

Further, there are several outstanding pull requests on GitHub to add 2.9.1 support. It was relatively trivial to update the libraries to 2.9.1, but they're not getting integrated. I haven't gotten any feedback about when these might get merged, or why Twitter is stuck on 2.8.1. I've heard through third parties that this is due to problems with Java interoperability with 2.9.1. Also, the custom SBT plugins used seem to, for some reason, break SBT's ability to cross-build.

Unlike with Ruby gems, using forked dependencies is non-trivial.

I intend to keep using Scala. This problem, however, is representative of the relative lack of maturity of the Scala ecosystem. I hope and anticipate this will be resolved in the future. As of now, they represent real problems that get in the way of delivering working code.

"Some of the nastiest nerds become lambs when they meet in person. ... If you see people often enough, and know that you’ll see them again pretty soon, calling them a retard on some forum becomes decidedly less gratifying."

Ha! I thought this was one of the most poignant statements in the post. Community engagement might be the solution to all the Scala backlash that seems to distract from making the language better and establishing best practices.

Ironically, it may turn out that what appeared initially to be one of Scala's greatest strengths, the close integration with the JVM, may be its undoing. As a richly multi-paradigm language Scala is complex enough as it is, but the various warts and leaky abstractions it inherits as a citizen of the JVM make it that much more perplexing.

Perhaps the idea that Java programmers would find it an easy first step into functional programming was incorrect, and in fact those bold enough in the first place to dip their toes into FP would prefer a cleaner and less compromised view of FP as provided by Haskell, OCaml etc?

I think there's a sizable audience out there that would like to use more FP but would also like to be able to integrate into their existing infrastructure. If you're already using Java it would seem much easier to sell "Hey, let's try Scala/Clojure for this, it can call our existing stuff" than it would be to sell "Let's rework substantial portions of our infrastructure so we can integrate with Haskell."

Yes, but these are the same people that run screaming after their first stroll through the function signatures in the collections library. The kind of people willing to man up for Haskell aren't going to be so easily discouraged.

There's nothing preventing compilation of Scala to other architectures. Indeed Scala compilers targeting the CLR and LLVM are under construction. Naturally that doesn't alleviate the problems baked into Scala to deal with JVM idiosyncrasies.

Having read this, I appreciate much more why Dispatch is the way it is. As a newcomer to Scala, I wish this wasn't the standard recommendation then. I was almost put off entirely when after 2 solid days I still couldn't issue an HTTP request. Fortunately I'm on the JVM and had other options, which I ultimately went with, but I was trying hard to embrace the Scala way and just coming up totally frustrated. If a simpler library were the starting point and Dispatch used for more advanced cases (again, as community recommendations), I posit there'd be a lot less vitriol.

But that's pretty much every Scala library right now. Every time I go out to find something I need it is always syntactically and documentationally complex/lacking and I wish that some of the visual simplicity that is available in things like python/ruby were available. After all, it is an incredibly versatile language right? So why is everything done in seemingly the most academically challenging way rather than in a way that allows developers to actually get shit done quick and go home for dinner. Any why do libraries get obsessed with having 2 or 3 ways to do something without documenting any one of them well enough or having any indicator of which one is better and if one is, why (Lift is particularly bad here though it is a fantastic tool once you get to grips with it).

Excellent response. I never understood the hate towards Dispatch - I had a lot of fun learning it with the help of http://www.flotsam.nl/dispatch-periodic-table.html

If you dont want the fun simply drop down to the Apache HTTP client and use its interfaces. Dispatch is just a thin wrapper around the Apache client anyway.

Reply on news.ycombinator.com