Why did Microsoft invest so much in F#? | Hacker News Reader

Why did Microsoft invest so much in F#?

57 pointsMrMan60 comments

Was F# viewed by MSFT as a cheap way to capture more developer mindshare, by accident, harvesting efforts by the research group? Or did they deliberately try to fill a niche they identified beforehand from marketing feedback?

F# is cool, but C# 4.0 does not seem to lack much, feature-wise. If anything the MSFT ecosystem seems to lack supporting tools, not language features. Why another language from scratch? Why not more effort porting an existing language to the CLR?

Also, IronPython and F# projects seem to demonstrate that it takes a long time to make a credible language for CLR/Visual Studio. A grammar for your language, a compiler, ok. How long did it take? But then you realize oh no, VS support will take many man-years. Whoops.

View on HN

F# is the place where you can unleash the power of your core developers and make objects fully usable by an army of C# developers.

When I used it no one wanted to learn F#, but if I put a nice OO interface on top of whatever I wrote, packaged it as a DLL the other devs would be happy to use it.

F# isn't just cool, it will make you write better code, faster. The zen of it makes you follow a better path. Think about how many bugs go away when you make all sides of an 'if' 'else' block return the same data type.

Think about how much more sense returning an out parameter as a tuple makes. I'd much rather write

  let value = match dict.TryGetValue("foo") with
              | (true,val) -> val
              | (false,_) -> "default"

Than

  string value;
  if(!dict.TryGetValue("foo", out value)){
      value = "default";
  }

Also, if you're using a match statement and forget to handle the false case, the compiler will scream at you. In C# you're on your own. Also, in C# in this case you can't even use var because you're not initializing it.

The other thing about F# is that it's FAST, fast like beating C at the burrows-wheeler transform.

FAST? It's still .NET. Regarding you example, I'd much rather write

  value = dict.get('foo', 'default')

in IronPython.

I'm not familiar with python - does it have some magic global sentinel value to indicate "not found"? Or how do you handle that case with that 1 line of code?

Secondly, saying "it's still .NET" is similar to saying "it's still x64". The F# compiler does a lot of transforms on code that the C# compiler does not. In some cases, this can result in better IL and better JIT'd code. Also, having inlining support at the compiler level can be very handy sometimes.

Yes, good points MichaelGG, one of the really nice things about the F# compiler is that it can inline higher order functions, so if you pass a function to another it may create a custom piece of code with your passed function inlined. I wasn't sure if you were referring to C style inline support, which C# does have (in the .NET JIT). Like C the inlining in F# is just a suggestion to the compiler.

So you can write code that is optimal from a design point of view (abstractions, etc) and the compiler will turn it into optimal code from a performance point of view.

See http://flyingfrogblog.blogspot.com/2009/07/ocaml-vs-f-burrow... for a much better explanation.

I don't think that F#'s inline is just a suggestion. Hat types (^a) aka statically resolved type variables, require inline methods to actually be inline. Same way you can't have inlined recursive functions.

You can nest a recursive function inside a non-recursive one and inline that.

> I'm not familiar with python - does it have some magic global sentinel value to indicate "not found"?

No, it has a different method for failable search. The purpose of `get` is specifically to handle "provide a default value in case the key is not found".

> Or how do you handle that case with that 1 line of code?

dict[key]

You can also use defaultdict[0], or define the __missing__() method[1] yourself on a dictionary you receive from someone else's code. In either of these cases the "not found" value will be whatever you decide it should be.

[0]: http://docs.python.org/library/collections.html#collections.... [1]: http://docs.python.org/library/stdtypes.html#dict

Upvoted because that's good code reuse in the scenario that you only want to store a default value, and I can see why you replied to the commenter with it. It doesn't necessarily work so well in the scenario where you want to perform some operation if there isn't a value. Or if default needs to be computed. I'm assuming that the idiomatic way of doing this in python would be:

try: value = dict['foo'] except KeyNotFound: value = do_something_interesting() dict['foo'] = value

If you could pass a function in as a default of course this point is moot!

See collections.defaultdict in the standard library -- http://docs.python.org/library/collections.html#collections....

> It doesn't necessarily work so well in the scenario where you want to perform some operation if there isn't a value.

No, but that's not what the example he replied to does.

  namespace Org.DictionaryExtensions =
    type Dictionary<'T> with
     member x.get(key,defaultValue) = 
       match x.TryGetValue(key) with
       | (true,val) -> val
       | (false,_) -> defaultValue

There fixed that for you, now you can do, dict.get("foo","defaultvalue") in whatever CLR language you like. :)

As for fast and knocking .NET you're not seriously suggesting python are you? Runtime method lookup is not exactly what I think when thinking fast.

I wasn't knocking .NET; I had the idea that coding something in C# or F# would result in the same IL output, which, after reading MichaelGG's reply, I realized was completely wrong...

Regarding Python speed, I know it's pretty slow, especially the IronPython implementation.

  (setf value (gethash "foo" dict "default"))

Hmmm... I'm slowly being convinced.

> let value = match dict.TryGetValue("foo") with > | (true,val) -> val > | (false,_) -> "default"

groan

    let value = findWithDefault "default" "foo" dict

    case M.lookup key dict of
         Just v -> v
         Nothing -> "default"

(and for anybody who wants to whine about parameters order, please remember — or discover — that partial application is extremely frequent in Haskell and MLs, and try to read the parameters order in this light, you will be enlightened)

dict.TryGetValue is an odd example here -- idiomatic F# for a map lookup would be:

    let value = defaultArg (Map.tryFind "foo" dict) "default"

    match Map.tryFind "foo" dict with
    | Some v -> v
    | None -> "default"

> dict.TryGetValue is an odd example here

I sure as hell hope so, but that's what fleitz used.

From my own personal experience, F# got Microsoft into a lot of hedge funds, prop shops, and other places with math/physics staff that like functional languages that would otherwise have been Unix-focused.

That doesn't sound right. I don't think F# is a commercial success.

Can you give me one example of a hedge fund that uses F#?

In my experience, some funds may keep their technology choices close to their vest.

A lot of Smalltalk programmers / NeXTSTEP programmer can vouch for that.

Jane street already use Ocaml, so they are probably lost to Microsoft. But that at least prove the presence of a market. Microsoft may want a share.

F# is used in quite a number of firms, especially in the Chicago area I am aware of a handful of companies. Initially it was used just for prototyping/modeling type stuff, but more and more it is seeing the light of day in production. It will never see the adoption scale of Java/C# but that isn't the point or the goal.

It makes sense as well if you follow the history of most of these trading firms. Most of them started out just manipulating excel spreadsheets. Then brought in vb macros and such and then from there some diverge into java or go straight to some functional programming solution like ocaml with jane street, but others stay tied to msft with C#. Then F# comes along and allows intelligent developers at these msft firms to express themselves more clearly when dealing with the type of math problems they are solving.

If the firm is using C# I am almost certain that F# is in the mix there as well.

The F# user groups in London are filled with people working in finance who use F#. One or two people using it for science or startups (data analysis) as well. Looks to be a fair amount of interest in using it as a language for side projects for the people I spoke to there, while MS are not pushing it as much as C#, they do support its use in the banks.

Credit Suisse has about 100 people in their algorithmic trading and risk management group using F#.

Thanks - I just finished at CS and was about to cite it as my example.

As NeXT proved, you can have good sales to high-end finance firms without being commercially successful.

Lots of hedge funds used MS stacks even before F# - I'm guessing the biggest reason is staff skill set both of the early developers and the other tech people. Most hedge funds will only have a single IT guy to maintain app servers (exchange, etc.) so it makes sense for the trading servers to be Windows as well to avoid having to hire a Unix sysadmin.

Why F#? Well, it's a great strategy for Microsoft to keep a language that certain geeks want to code in. I've been on .NET since before the first Beta, and I would not stay on .NET for many projects if it weren't for F#. Writing in C# is just too painful. F# makes developing for .NET a pleasure.

C# is not even close to F#, feature wise. C# and VB have taken the path of picking the "sweet spot" of features, and adding special compiler magic to implement those exact things. So, on some comparisons, C# looks just great. But actually going off the blessed path means a lot of pain. Here's a quick list of why F# is nicer:

  - Vastly more lightweight syntax
  - Far more advanced type inference
  - Sequence expressions (and list/array comprehensions)
  - Modules
  - Hierarchical code (nest modules in modules, functions in functions)
  - Interactive mode (FSI)
  - Custom dynamic support (it provides an operator, you provide the implementation)
  - Inlining support (handy for type tricks and great for performance)
  - Recursion support (either tailcalls, or the compiler rewrites your functions to loops)
  - Object expressions (create an object implementing an interface on the fly)
  - Workflows (especially for async code)
  - Full quoted code (C#'s Expression<T> is pretty weak - again, it was just made to enable certain translation to SQL scenarios)

That's just off the top of my head.

Oh, and it still has nearly all of C# imperative/OO features. I can pass addresses, allocate off the stack, use pointers, and so on. On the OO side it has everything except the ability to create new "protected" visibility members.

Basically, even if you know nothing about functional, and just code in an imperative and OO style, F# is better.

Type Providers in the next version of F# should be another impressive feature. This will really change the way F# interoperates with data and Web APIs. Type providers make everything available, strongly typed, from intellisense with no visible glue code.

http://bit.ly/bfWDkA

Functional programming languages such as Ocaml/F# offer many things that are not in imperative languages such as C# (and arguably vice versa). In the spirit of ``the right tool for the job'' the .net platform needs a functional language to solve certain problems effectively. In this sense the value of F# is far more important to the .net framework than e.g. IronPython.

Very interesting - F# fills a fundamental need that cannot be supplied even by C# 4.0 with its post-modern featureset. Thanks for that opinion.

MSFT had a push towards basic research in the last decade. To do that they needed to hire PhDs and give them some control over what they wanted to work on. Most CS PhDs these days want to work in a language with Hindley-Milner type inference, so they created F#.

CS PhDs who work in the area of programming language design.

Note also that Microsoft employs Simon Peyton-Jones, one of the foremost contributors to the Haskell programming language, so it's not clear that they needed to create F# for language researchers. Although, as Don Syme says in the interview quoted by TomOfTTB, an ML type language mated much more neatly with .NET than Haskell did, so you might still be right.

I was at a talk once by Ted Neward. He mentioned that many of the C# features we have today are because the F# team needed them in the CLR. C# owes a lot to F#.

If Clojure and Scala are popular on the JVM, then why not F# on .NET? And you can write F# code in Emacs, as God intended.

> Why not more effort porting an existing language to the CLR?

It didn't work very well. The mismatch between your favorite language and .NET caused 2 problems. First, the procedural/OO style required to interop with .NET bled into the rest of the program. Second, performance sucked for dynamic types, tailcall, small object allocation, etc. The result was you'd write a program in your language that looked similar to C# but ran 2X slower.

Scala and Clojure get around this by being new languages that try to match the JVM platform better. Plus they provide fat libraries so you don't have to see Java objects too often.

> F# is cool, but C# 4.0 does not seem to lack much, feature-wise.

C# doesn't have algebraic data types, pattern matching, currying, active patterns, metaprogramming, asynch workflows and global type inferencing. There's more, but you see my point.

MSR have had a few profitable collaborations with their .net language implementors. The generics system in c# for example was originally developed by Don Syme (who also worked on f#) and Andrew Kennedy. I think once you've had a good example of a successful collaboration between the two groups you're more likely to see future cooperation. The local type inference in c# (the var keyword) I believe was also improved heavily by discussions with MSR.

With regards to your 'from scatch' question I think f# is designed in order to bring useful functional programming techniques to the .net community. This isn't necessarily possible to do with Haskell because of the difference between the way function calls work in lazy and strictly evaluated languages.

Its worth noting the other aspect of the f# story. Ocaml is a huge influence on the f# language design. It was originally pushed during the 1990s as a language that had combined functional programming with pragmatic compromises - for example OOP and the existance of an efficient implementation. Unfortunately the Ocaml implementation suffers from poor performance in multicore environments, particularly constrained by its garbage collection system. This wasn't a huge problem in the mid 90s, but its looking like a problematic design choice with hindsight. Since f# builds on the .net VM it doesn't have this multicore issue.

Since the promise of the CLR was any language one platform they went out and built several different types of languages to see how well it worked. F# actually attracted developers so they decided to keep it.

Note that F# began life as Ocaml on the CLR, not another language from scratch. Then it picked up "light" syntax and then moved to light as the default syntax. Also note that F# has been around for a while now, so yeah C# 4 isn't bad but C# 4 wasn't even on the horizon when F# got started.

Is F# still really just another ML flavor? In any way that is meaningful in the real world? Another poster on this thread asserts that F# adoption is happening in finance, by FP practitioners who would <i>otherwise use unix!</i> My impression (perhaps flawed) is that the main audience for F# are C#, Java and other programmers for whom F# is indeed their first foray in FP. It seems like the imperative/functional hybridization of F# mirrors the Scala effort to market itself as a FP language that imperative/OO folks can understand. I am not espousing a point of view, I am asking for insights about how a large firm like MSFT sees the dev tools market, not trying to poke a stick into other folks' ideas about what the best tools are.

I would say it is another ML flavor. It has ranged much further from SML than other SML derived languages have. To me it is about like calling clojure or Dylan lisp.

I love that not one person has made a rude comment about Microsoft. I know I am adding noise instead of signal here, but I want to say: Congratulations HN! Keep it up!

Interviewer: I've heard that there was a project where Microsoft started to integrate Haskell on .NET and then it was replaced with the F# project. Is that true? If it's true, why?

Don Syme: That's a small part of the sequence. The visional design of the .NET platform was very much expected to be a multilanguage platform from the start. Right back in 1998, just in fact as our research group in programming languages started at Microsoft and I joined the team and then other 10 of us joined the team, we were approached by a guy called James Plamondon, who started the project called Project 7, which was about getting 7 academic and 7 industrial programming languages on each side to target the .NET common language runtime and really check out if it was good enough, to see if design changes could be made early on in the design process of .NET to make sure it was good enough for a range of programming languages.

Some of those design changes were made, like tail calls were, for example, were added in the first version of .NET and that was a very interesting project because they gave a lot of way to our group and researchers at Microsoft to make connections between the academic programming world and .NET. We have seen that there are a lot of people working on .NET over the years, and also let our group work directly on .NET with regard to .NET Generics and other proposed extensions to .NET - we got these researchers engaged with the system.

We sort of knew there were opportunities to do new languages, but to contribute to existing languages like C# and also to do new languages in this kind of context. We talked a lot about doing a systems programming language of some kind, something that would effectively sit between C and C#, that you'd be able control memory usage and maybe get safety properties at the systems programming level.

We ended up not pursuing that, although people are still interested in doing "managed" systems languages, say languages you could drive a whole operating system in or write large portions of Windows in. Then, at the other end, we are interested in doing these expressive functional languages. Haskell .NET was something we looked at closely, we always had in mind to do NML as well, there is a project called SML.NET, which is a very good implementation of standard ML for .NET with a Harley optimizing compiler and the like. We learned a lot from that - a great project!

==== The paragraph below is the most relevant to the Question ====

I took what I learned from .NET Generics and saw that there was a chance to do a ML like language fitted very closely with .NET. During this time we had a go doing Haskell for .NET, we actually got a long way in doing that, but in the end there is quite a lot of dissonance between Haskell and .NET. The purity is one aspect of that so you are writing monadic code whenever you use the .NET libraries, which would be perhaps unusual, would lead you to writing more monadic code than you would like. Also, Haskell didn't have any tradition of adding object oriented extensions to Haskell.

====================================================

There was one project called O'Haskell, which was interesting, but there was something about Camel which had a tradition of taking a call language and then making extensions to it, changing it, experimenting it, but taking a core paradigm that works very well and then making it fit to a context. In a sense, in Haskell there are too many points where the model wasn't going to work in the context of .NET and you get too much dissonance across the interoperability boundary in particular - actually there are some serious challenges in making it run well at all. That was definitely an interesting project and I think a few other people have tried the Haskell .NET, but we didn't continue with the project.

Source: http://www.infoq.com/interviews/F-Sharp-Don-Syme

Just to add to the historical perspective... One of the Project 7 languages was a Haskell dialect named Mondrian. If memory serves, this was brought to production as a scripting language back in the .NET 1.x days, but it never really got much traction and is no longer available. The same people have continued to work on Haskell for .NET, but only as an in-house project (see http://kahu.zoot.net.nz/).

My understanding is that it was heavily invested into as an internal tool for MS's own use way before it was touted as an external product.

What if F# is somebody's pet project? Microsoft might have invested in F# because the supporters inside Microsoft had the political capital to make it happen and not because there was a real product strategy.

Some things about F# are consistent with the pet project theory:

- The language was developed by a team in Microsoft Research, not by a product team. If a product team thought there was a good product strategy for the language, they would have taken it over.

- Some of the reasons given for F# are not a direct customer benefit. Example: show that CLR can support languages other than C#.

- Given the question here on HN, it's not obvious to some people why Microsoft invested so much in the language.

I raising the question about why Microsoft invested in and released the language and not making a statement about the quality of the language itself.

A friend of mine who still works at Microsoft confirmed that the pet project theory is partially correct.

> The language was developed by a team in Microsoft Research, not by a product team. If a product team thought there was a good product strategy for the language, they would have taken it over.

Luke Hoban's F# product team in Redmond develop F# with the research team in Cambridge UK continuing to evolve it.

> Some of the reasons given for F# are not a direct customer benefit. Example: show that CLR can support languages other than C#.

F# was not the second language after C# to run on the CLR.

> Given the question here on HN, it's not obvious to some people why Microsoft invested so much in the language.

The investment is tiny for Microsoft and has already paid off by drawing people into the Windows platform who then buy Visual Studio 2010 in order to use F#.

The explanation I've heard (from a talk by Ted Neward) is that going increasingly multicore is where most future performance boosts will come from.

Since pure functional languages are easier to parallelize, Microsoft is interested in languages like F# for their performance on multicore systems.

> Since pure functional languages are easier to parallelize.

Purely functional programs have a lot of problems with parallelism. If you want to parallelize your F# code effectively, you write Cilk-style code with heavy use of mutation and not Haskell-style code with purely functional data structures.

Read this: http://flyingfrogblog.blogspot.com/2010/06/regular-shape-pol...

I'm the James Plamondon mentioned below.

Having been out of Microsoft for quite a while now, I can't answer the question authoritatively.

However, having helped make .NET language-neutral, I can offer these two specualtions: (1) After a decade of making the.NET Runtime and Visual Studio.NET language-neutral, implementing F# required much less investment than you might think (although I'm sure it wasn't "cheap"). (2) When it comes to cloud computing, Microsoft is "all in," and functional languages are ideal for cloud computing (being naturally parallel - see the Church-Rosser property of the lambda calculus). This is the main reason, I suspect.

> IronPython and F# projects seem to demonstrate that it takes a long time to make a credible language for CLR/Visual Studio. A grammar for your language, a compiler, ok. How long did it take? But then you realize oh no, VS support will take many man-years. Whoops.

IronPython has had working (albeit beta) VS support for nearly half a year now.

ironpython.net and get IronPython toools for Visual Studio.

I know they exist, thanks. I am asking only about the motivation for fostering a new language for the CLR, and the costs involved. I am proposing that building in tool support is, while necessary for adoption, costly.

Another anecdotal aside - small teams seem to throw together Eclipse-based tools with little cost. From a UX point of view, Eclipse is not provably "better" than tools built using Visual Studio, right?

The problem with Visual Studio tools are COM, and a very high overhead to get something simple working.

There are a lot of magic incantations you need to know to write VS plugins. If you don't get the magic words right nothing happens.

If you have programmers on staff who know Visual Studio inside out, it's pretty easy to get stuff working. Like, say if your company had the team who wrote Visual Studio.

Also, the tool support for F# sucks, basically the only thing Visual Studio will do for you in F# is intellisense.

Drag a dbml file into an F# project? Nope, sorry doesn't work.

xaml? same thing.

ASP.NET? yeah right.

If you want tool support in VS you better open up the project file and start adding it in.

This is a key response. I am more interested in why MSFT would spend more resources creating more languages than on streamlining the facilities for creating languages. Maybe dog-fooding these things over the last 8 years or so has spurred, or will spur product groups and/or MS Research to put as many man-years of work into the meta-tools so that languages can be launched with much less effort. I am impressed, needless to say with F# itself, and I have learned a lot from this thread that I did not know about it, but "what is F#" is not the point of my question.

Instead of developing a language similar to Java (C#) or similar to Ruby/Python (F#) or investing in companies / ideas they were not successful at (Social Network / MSN 10 years ago), I think they need to bring back the "Bill Gates" innovative spirit... what's the language of choice 5 years from now? what are the hurdles most developers are going through today and how are they solving them?

F# is not very similar to Ruby/Python, it's closer to OCaml.

F# is not like Ruby/Python at ALL. F# is statically typed for one, variables are by default immutable, let me give you an example, tell me what you think the following prints?

  let somefunc() =
    let i = 10
    i = 11
  printfn "%s" (somefunc().ToString())

it prints "false", not 11.

F# is a lot like ocaml, which it was syntax compatible with for many years.

Most developers in the non-web sphere are dealing with multi-core and async issues. F# solves these problems very elegantly.

yes, very true. closer to Erlang or Caml instead. I guess I see the value for a functional programming language written on top of the CRL

Reply on news.ycombinator.com