Hacker News Reader

Comment by runT1ME | Hacker News Reader

How does asynchronous futures end up being more complicated or in any way worse than the blocking threaded approach?

    for { 
       user       <- asyncGetUser(1)
       company    <- asyncGeCompany(user.companyid)
       longresult <- asyncProcess(company.getSomething)
    } yield longresult

View on HN

His issue seems to be the potential for modifying or referencing mutable state where in the yield block (or map or flatMap or whatever).

It can be a problem. In Akka actors, referencing sender() from a future will be unpredictable because sender() could've changed in the mean time.

I think there are three strong solutions which address this problem:

1) Immutable state. Solves this problem completely but accidental capture remains an issue.

2) Hiding mutable state within actors. I hesitate to present this as a general solution though since it really requires going all-in with actors (I don't consider that to be a bad thing, necessarily).

3) Projects like Scala Spores [1] aim to tackle this at the compiler by capturing mutable references and executing async closures in an immutable environment and prohibiting accidental capturing. IOW, turning accidental capture into a compiler error. I'm excited about this one.

[1] https://speakerdeck.com/heathermiller/spores-distributable-f...

You can (and should) use Futures independently of Akka actors. You're correct, hidden mutable state with Actors can be a problem, as with mutable state in general.

Absolutely, but people make mistakes.

It's also somewhat unavoidable when you're dealing IO, unless you stick to .pipeTo(self)

Yep, that's very nice code, except:

    user = getUser(1)
    company = getCompany(user.companyId)
    longresult = process(company.getSomething)

is a) simpler (because that's what your normal code looks like), b) performs exactly the same (as fibers basically do the same thing, only transparently), and c) retains context (like ThreadLocal variables).

So if that was the only way, I'd say, fine. But once you have lightweight threads, they are always preferred (even Scala now has its own poor-man's lightweight threads in the form of async/await).

> simpler (because that's what your normal code looks like)

Actually, no. My code will always look like the for comprehension be it async or not, because my 'get' methods will always return an Either, Option or some other monadic context to represent the computation succeeding or not.

> performs exactly the same (as fibers basically do the same thing, only transparently),

Yes, I agree.

> c) retains context (like ThreadLocal variables).

Threadlocal variables are almost always code smell. If you find yourself using them, there is almost certainly a better way of accomplishing the same thing.

I don't see how having a Fork/Join threadpool that 'powers' the futures would be any slower than lightweight threads either.

When I learned to program (back in the 80s), I learned that if you want to tell the computer to do operation X and when that's done, do operation Y, you just write both statements one after the other. If you're comfortable using for comprehensions to achieve that same goal -- that's great. To me it seems that your code simply replicates what a thread does. If you have a problem with your thread's implementation -- fix it. If you want the compiler's help -- use the compiler to write the for comprehensions, or monads for you. Personally, I don't think you should need monads for "do X then Y", and neither does Scala, by the way. Scala says that you only need them if you want to say "do X then Y, but don't use OS threads".

I wanted to ask you, what about running multiple operations in parallel? I've long been contemplating async/futures vs lightweight threads and yet to have come to a conclusion.

With futures I can say, start operation 1 and operation 2 in parallel, then chain a callback to execute using both pieces of data, saving me some latency of doing the operations serially. How do you do this using quasar? Now that's a trivial example... what about arbitrary dependency trees? This falls out naturally with futures but I don't know of a nice way to do this with lightweight threads. e.g. 1 operation branches off into 5 other parallel operations which then chain some of their own additional callbacks for processing before finally bringing all the results back together, perhaps to form a json response. Does this example make sense?

With lightweight threads you can spawn as many fibers as you like. Creating and starting a new fiber is basically free. You can start fibers and join them, in any dependency tree structure.

Of course, you can keep using futures (what I call semi-blocking API), only futures that block the fiber rather than the thread, when you join them.

Okay so to run operations in parallel you have to go back to futures, and for more complex dependencies you would need callbacks/transforms. So this means with fibres you could use the simpler synchronous model for serial operations, but future model for parallelism, i.e. a hybrid model. My thoughts are that it might be simpler to adopt a single model rather than two. On the flip side you could argue that with fibres you don't need to use the more complicated parallelism model all the time and only when needed.

If you want to run operations in parallel and then "join" them, you need some joining mechanism. A fiber itself can be joined and return a value, so in Quasar, Fiber implements Future. But that's semantics: with fibers, if you want to run operations in parallel, you spawn more fibers; if you then want to join those operations -- you join the fibers.

Of course you can have long-running fibers that interact with one another through channels. See: http://blog.paralleluniverse.co/2014/02/20/reactive/

Thanks, this is the kind of answer I was looking for. I'll have a read once I'm on a bigger screen.

Seriously, just use scala Futures. Here's one of many super easy ways to do parallel computations in scala: val list = getItemsToProcess() //List[SomeObj] list.map(so => Future { processorHeavyMethod(obj) }) //happening in parallel now val finished = Future.sequence(list) //Future[List] finished.await

Reply on news.ycombinator.com