Hacker News Reader

Comment by nutate | Hacker News Reader

F# is a real breath of fresh air in comparison to something like Scala. It's direct ML heritage really shows, also just diving in with an IDE (like Xamarin, or I suppose Visual Studio) is super easy.

I see it as the future of pop-functional programming. For example look at the way it handles type inference w/ JSON parsing. Compare that to what you have to do to parse JSON in Scala. It's subtle, but a major usability win.

View on HN

Scala is getting really interesting lately though with projects like Spark and SparkSQL. Spark takes a Scala AST tree and computes a plan on it in order to distribute it to a cluster of possibly thousands of nodes and then uses transformations on that plan, similar to how an SQL query optimizer works, to make the plan more efficient. The cluster compute benchmarks on it are crazy good.

The only problem I have with Scala is that it has so many features that it tends to have a steeper learning curve than most languages. However, this is mitigated somewhat in that it lets one switch between object and functional depending on what's more convenient or performant, thus making it easier for programmers who are not highly skilled in functional programming to pick up.

Would there be a F# equivalent to that Scala/Spark thing?

I'm not in the know here, but maybe M-Brace is vaguely similar. Unsure about the optimization part. http://www.m-brace.net/

so I googled and saw this.

one could say they're... looking into it...

https://careers.microsoft.com/jobdetails.aspx?jid=170944&pp=...

extract:

> Inspired by Spark, we are building OneNet, a distributed functional programming platform based on F# that allows programmer to build distributed system at >3x productivity. OneNet offer similar programming model and extensibility of Spark, but goes much beyond in distributed functional programming to offer additional capability and performance (e.g., in-memory sharing of instantiated object with concurrent operation, use of both managed code & unmanaged code in functional programming, use of model and/or key value store, multi cluster with EDGE + cloud, private + public cloud compute, use of GPGPU/FPGA in cluster)

Yeah, I did a presentation at the DC Area Apache Spark meetup... slides: http://www.slideshare.net/RichardSeymour3/2015-0224-washingt... associated blog post: https://www.endgame.com/blog/streaming-data-processing-pyspa...

I've done a bit of scala spark as well, and my initial thought was prototype in pyspark and then rewrite in scala if necessary. Just this week DataBricks announced they are working on changing some of the data structures behind RDDs to save on unnecessary java object creation hubbub https://databricks.com/blog/2015/04/28/project-tungsten-brin...

That and the SQL compiler thing seems pretty darn awesome. Spark has the nice benefit of being plug and play (w/ a joyful time of compiling and deploying) with legacy HDFS/Hadoop systems. That alone will keep it in toolboxes for a long time to come.

>Spark takes a Scala AST tree and computes a plan on it in order to distribute it to a cluster of possibly thousands of nodes

Unless Spark has changed dramatically in the year since I used it, that's not really how it works. You lazily construct an expression graph of RDD operators but the actual Scala code (the functional payload of e.g. flatMap) doesn't get analyzed. Are you talking specifically about Spark SQL's code generator?

>The cluster compute benchmarks on it are crazy good.

...and also carefully chosen to show off the niche cases where Spark shines. More commonly, you'll encounter crippling garbage collector stalls leading to mysterious and undebuggable job failures.

Spark has a very clean API but the implementation (at least for Spark versions 0.8-1.0) is still prototype quality.

Yep!

So basically you're talking about the design and limitations of https://databricks.com/blog/2015/04/13/deep-dive-into-spark-... I take it?

The implication of this work I thought is that it could be further expanded to other languages and DSLs. However, Spark's SQL generator still being very JVM-dependent and this optimizer generating bytecode kind of makes it pretty specific to JVM-supporting languages only. This would probably leave out Haskell and Erlang / Elixir in the short term, which is probably where I'd expect to see a different perspective on the whole data analytics front. We have Datomic, sure, (and I guess Clojure is enough for a lot of folks) but it'd be nice to have something other than "we want to make Hadoop... BUT BETTER" as a lot of the motivations.

I did a toy comparison of ML-derived languages running on Linux the other day (http://thebreakfastpost.com/2015/04/22/four-mls-and-a-python...) and found F# already simple to get along with and fast to run, at this beginner level. It could become a really interesting option.

Thanks, very interesting. Do you happen to know how large OCaml binaries get if you use Core.Std?

No, I haven't even downloaded Core. I am more ignorant about OCaml than any of the others. I'd love it if someone else here could weigh in on the pros and cons.

Good writeup. I posted it to HN: https://news.ycombinator.com/item?id=9463254.

Maybe you could try out Haskell also? I would gladly assist if you'd be in need of any.

Also interesting is their F* language: http://fstar-lang.org/

"F* is a new higher order, effectful programming language (like ML) designed with program verification in mind. Its type system is based on a core that resembles System Fω (hence the name), but is extended with dependent types, refined monadic effects, refinement types, and higher kinds. Together, these features allow expressing precise and compact specifications for programs, including functional correctness properties. The F* type-checker aims to prove that programs meet their specifications using an automated theorem prover (usually Z3) behind the scenes to discharge proof obligations. Programs written in F* can be translated to OCaml, F#, or JavaScript for execution. "

It's open source too: https://github.com/FStarLang/FStar

I'm going to write a package manager and call it F* ck

There's already a mocking lib called Foq: http://trelford.com/blog/post/fstestlang.aspx

Gotta love automatic theorem proving. I read the Shen language book on "logic, proof and computation" which got me into the idea of a theorem prover inside of your normal language.

Have you worked with it? I love the idea of a fully powered F# without the kiddie gloves. F* looks pretty neat.

If F# is of interest to you then check out http://www.fsharpworkshop.com/ + https://github.com/ghuntley/fsharp-workshop and finally http://fsharpforfunandprofit.com

Did you just invent a term? "Pop-Functional"... I like it!

Starts singing

make my func the p-func / I want my func uncut...

and i even hear marilyn manson doing a dirty version of it.

"tainted func"

Marc Almond maybe.

I hope so.

You would think something like OCaml would fill in that niche with its wider availability.

They do seem like very similar languages. I imagine in most cases your choice would have effectively already been made for you by the context -- e.g. whether you wanted the option of fast native binaries (OCaml) or deep .NET integration (F#).

There's a very cool implementation of (multi-threaded!) OCaml for the JVM: http://www.ocamljava.org/

I'm upset this wasn't called Joe Camel. And marketed towards children.

Yeah, but that's because there already is a JoCaml: http://jocaml.inria.fr/

OCaml doesn't come with the ecosystem that F# does. It isn't about availability, but APIs to play with.

I'm not sure I buy the ecosystem argument. We've recently seem other languages come from out of nowhere, with no ecosystem, to become major languages (Golang, for example).

I don't think there's really anything stopping OCaml from being more widely adopted, except for lack of attention from trend-setters.

Well, one is sponsored by and used in production by Google, and the other is maintained by the French Institute for Research in Computer Science and Automation.

Good point.

Go gets a lot of buzz on places like HN, but that's a long way away from being a major language.

Rob pike and company are also systems folks; the language supports the ecosystem they are building, not the other way around.

I don't agree, it's 'direct ML heritage' is quite superficial, as it omits ML's most powerful feature, functor modules.

Scala however has all kinds of super powered functional programming features that F# is missing, implicit parameters for type class constraints, ADTs, higher kinded types, etc.

I know it doesn't say much, but having never coded in ML and having tried F# briefly (I code mostly C#, and it'd be easy to have mixed F#/C#), I think Scala has much clearer syntax, or perhaps it's just more familiar for someone who's coming from imperative languages like C++ or C#.

  case class Fruit(name: String, variants: Set[String])

  val fruitJson =
    Json.parse(
        """{
             "name": "apple",
             "variants": ["cox", "braeburn"]
           }""")

  val fruit = fruitJson.as[Fruit]

Doesn't seem to be too painful to me.

I agree that F# is syntactically closer to ML (though Scala is closer to ML with its module system which F# gave up completely).

Tell me what happens with that technique when your json has > 22 keys. ;)

(Edit) this is apparently working in the latest scala 2.11 But it killed my adoption of scala for real work early on. https://issues.scala-lang.org/browse/SI-7099

It works without issues.

I never have seen a use-case where this would have been an issue. Sure, it "looks" unfortunate, but there is only so much you can do when running on a completely uncooperative runtime.

If devs need a typed representation of unlimited length, they provide a HList and move on.

Such a representation will replace the length-limited tuples in a future version of Scala.

In this base case, no, it isn't too painful. But once you get to larger and more complicated structures, F#'s type provider system can be a tangible timesaver. Especially if you're doing exploratory programming.

TBH, I think the JSON example might underrate things a bit anyway. Where type providers really start to feel impressive is when you get to tricks like being able to noodle around inside undocumented (or poorly documented) COM interfaces with the help of CodeSense, or getting red squigglies in your editor (and compile time errors) when there's a problem with an SQL query.

F# isn't all roses, of course. I really wish it had typeclasses, and discriminated unions are less powerful than Scala's case classes in some tangible ways. But from a purely pragmatic perspective, my (almost certainly biased) sense is that I can usually Get Things Done™ with less ceremony in F#, and type providers are a shining example of that pragmatism.

Well, then I think JSON is a petty bad example, because pretty much no service provides some schema-like specification of their protocol in JSON.

If there is one thing I learned using type providers it is that maintaining your own "description"–be it as sample data or a schema file–loses most benefits of using type providers in the first case.

On the other hand, want a type provider for X in Scala? Just write it.

Stuff is still moving around because Scala devs want to get the meta APIs right first, but if adapting things from time to time is not an issue, then go ahead.

Learn Scala before you say something about it.

Reply on news.ycombinator.com