Comment by saryant | Hacker News Reader

Comment by saryant | Hacker News Reader

I started using Scala ~2.5 years ago for my undergrad thesis. At the time I didn't really know what I was doing and essentially wrote Java with map/reduce on collections.

I've been working steadily in Scala since and currently do so professionally. I'm a fan, especially with the Akka actor framework.

Lately I've been playing with Play, Akka and ReactiveMongo. For-comprehensions offer a terrific abstraction for dealing with asynchronous code. My current side project is "Street View for Flights" implemented in the above stack. Lots of calls to third-party REST APIs after checking if data has already been retrieved and inserted into Mongo. Code like the following allows for controllers to be asynchronous while staying readable:

  def byCode(airlineCode: String) = Action {
    Async {
      for {
        airline <- Airline.findByICAO(airlineCode)  // returns Future[Option[Airline]]
      } yield airline map { a => 
        views.html.airline(a)
      } getOrElse NotFound
    }
  }

The advantage for me is that those futures could be redeemed either by finding the requested object in Mongo and on failure retrieving it from a third-party API while not blocking and keeping the code readable. Either way, the controller just gets a Future without blocking.

Other parts of this project make heavy use of Akka actors to feed server-sent events (sort of a compromise between Comet and WebSockets). Development has been great fun because Scala really allows me to write code at a higher level. Asynchronous, typesafe, nullsafe code with the compiler as my safety net.

The main problem I have with actors is that they are untyped.

What's the point of having a powerful type system like Scala's and then throw it out of the window just so I can use a distributed framework?

My understanding is that the new typed channels feature in the upcoming Akka 2.2 release will address this.

The macro feature introduced in Scala 2.10 makes this possible.

http://doc.akka.io/docs/akka/2.2.0-RC1/scala/typed-channels....

It's possible to make them typed with the stdlib[1]. Link also gives some reasons as to why they're untyped. Short answer is it started as a way to replicate Erlang in Scala[2].

[1] http://stackoverflow.com/questions/5547947/why-are-messages-...

[2] http://article.gmane.org/gmane.comp.lang.scala/12988

Scalaz has some nice (usefully typed) concurrency tools:

https://github.com/scalaz/scalaz/tree/scalaz-seven/concurren...

https://github.com/scalaz/scalaz-stream

Like everything with Scalaz, the tools are great but almost completely undocumented.

> Like everything with Scalaz, the tools are great but almost completely undocumented.

Exactly. I will start considering scalaz once its authors grow up and realize that the lack of documentation is not a sign of quality.

The maintainer and contributors have full-time jobs. On the one hand, part of that job is keeping up Scalaz for their own professional usage. On the other hand, no part of that job is documenting it for everyone else.

They could just have kept the code proprietary.

Really now? As if these guys get paid to do create and fix scalaz all day right? Like Oracle yeah? This is the story about every single open source project. You can choose something else, wait it out for documentation, or contribute it. Lucky you even have the source!

let's say you have more than one future:

    findAirline:        AirlineCode =>                Future[Option[Airline]]
    getFlightsForToday: Airline =>                    Future[Seq[Flight]]
    findAvailable:      Seq[Flight] => Constraints => Future[Option[Flight]

compose them so that I get:

    findFlight: AirlineCode => Constraints => Future[Flight]

I don't want `Future[Option[Flight]]` since Future has own error handling (using Try). No cheating by using scalaz monad transformer.

    def getOrThrow[T](x: Option[T]): T = x.getOrElse(throw new AirlineError)

    def findFlight(code: AirlineCode, constraints: Constraints) = for {
      airline <- findAirline(code).map(getOrThrow)
      flights <- getFlightsForToday(airline)
      available <- findAvailable(flights, constraints).map(getOrThrow)
    } yield (available)

No idea why you consider using scalaz to be cheating.

ah, getOrThrow! nice.

I'll readily admit that my Scala isn't 100% idiomatic, I'm still learning!

From my point of view and understanding, for is not good for this. Isn't it blocking? Say you have

  for 
     a <- forA
     b <- forB
     c <- forC(a)

it does not do a and b in parallel and then c, or does it?

It depends. Let's imagine forA, and forB are expressions that evaluate to futures of Ints. This runs them in serial:

   for {
     a <- forA
     b <- forB
   } yield a + b

This runs them in parallel:

   val fa: Future[Int] = forA
   val fb: Future[Int] = forB

   for {
     a <- fa
     b <- fb
   } yield a + b

(I dropped forC from your example as it has a dependency on the result for forA, and thus can't run in parallel.)

That forC has a dependency on a was the intention, so it needs to be run in serial.

I can't see the difference between your examples and so am confused why 1.) is serial 2.) is parallel.

At the risk of being trite, it's down to the order of evaluation.

You know that a for comprehension expands into calls to flatMap and map? So

  for {
     a <- forA
     b <- forB
   } yield a + b

expands to

  forA flatMap { a => forB map { b => a + b } }

Remember that forB creates and returns a future. Only when this future has been created will it start running.

It should be apparent that forB is not evaluated till forA's future delivers a value, and thus the future returned by forB will run in serial with the future returned by forA.

In the other example the futures are created outside of the for comprehension, and thus start running in parallel.

Ah ok, thanks!

Actors and the way Scala handles async is one of the reasons I wanted to get into using Scala on Android actually. The awkward way Android handles it is chock full of Java boilerplate and Scala's way is just much more elegant.

Another feature that got my attention was the ability to have a SQL framework that handled things similar to LINQ. I haven't tried it much yet, but Squeryl[1] has looked promising for being a Scala equivalent. I may have to look into Akka more though as that will easily take care of async SQLite calls as well[2].

[1] http://squeryl.org/introduction.html

[2] http://www.gamlor.info/wordpress/2012/05/async-sql-and-akka/

I used Squeryl in my thesis (admittedly a while ago) and Slick[1] on a more recent project. Like you, I had ample experience in LINQ and it's definitely the feature I missed most from C#. I'm a fan of Slick.

I'm also looking forward to async SQL calls.

I've been building some Akka-based projects for a client and both they and I have been thrilled with the results. They're a startup that's gaining traction and they brought me in to help with scalability. I won't go so far as to say that Akka is a magic bullet but it makes reasoning about scalability far easier, though in our case we're mostly dancing around the IO bottleneck.

[1] http://slick.typesafe.com/

My GSOC project for this year is an extension to for comprehensions to enable LINQ-like conciseness without LINQ's inflexibility. If it ends up being merged into the compiler proper (a pretty big if), it should really help to make writing complex queries a lot less painful.

That's excellent. 9 of 14 proposed projects were accepted

http://www.scala-lang.org/gsoc2013

http://www.google-melange.com/gsoc/projects/list/google/gsoc... (type "scala team" in "Organization" box

Squeryl doesn't allow you to drop down to raw SQL, so I'd suggest avoiding it.

It is also rather broken, and the author doesn't appear to care. I'd second the suggestion to avoid it.

That's a shame, on both accounts. It looks nice and the right way to abstract out SQL, but admittedly, I would not use it if it won't let you drop down to SQL for something that needs it.

Reply on news.ycombinator.com