Advanced programming languages
matt.might.net
matt.might.net
The people I know who do work in machine learning (Jon Kleinberg, Thorsten Joachims, and their students) work in straight-up C, not Haskell, as the author suggests. Most of the dominant methods used today, such as the SVM, boosting, and the dynamic programming methods used in genomics are (i) numerical more than symbolic and (ii) involve moving a LOT of data around, so you want control of your data structures. (Sometimes these guys compress in-memory pointers!)
I looked long and hard at Scala and decided that I didn't like what I saw. One problem is that, based on the JVM, type erasure is a big "broken window" that seriously damages the Scala type system. You can use generics for ~years~ in Java and never notice the problems that type erasure causes, but I find that 2/3 of the designs that I specifically want an advanced type system for do not work in Scala because of type erasure.
Another issue with Scala is that the interface with Java, particularly using collections, is awkward; people who want to work in the JVM but want concision might be happier with Groovy. Scala's Lispy-Lists are convenient for the pattern matching capabilities in Scala, but are much less efficient than the vector-based Lists that come with Java when your data gets big.
As for oCaml, the real excitement is in F#. You get the wonderful oCaml language and all of its features, plus you get an interface to the C#/.NET world that's much nicer than Scala's interface to the JVM. You've got access to the whole .NET framework base class library, plus all sorts of stuff that's been written for .NET.
I'd also put C# on the border between a "mainstream" and "advanced" language. C#'s generics implementation just added support for covariant and contravariant inheritance, which is one of the features that had me interested in Scala. C# has good lambdas, really neat stuff in LINQ, and "expression trees" which offer metaprogramming capabilities above and beyond any static language I've seen.
Note that there is a good open source implementation of the .NET framework, Mono, so you can develop in C# and F# and target Linux, MacOS and other platforms.
This, combined with specialization (compiling primitive-specialized versions of code with generic types to improve performance of, for example, using function types on primitive collections) will make it easier to write performant, numeric code in Scala than in any other JVM language (up to the limits of the JVM, of course).
The NLP communities at Stanford and Berkeley, which do a fair amount of machine learning, have a long history of working in Java (http://nlp.stanford.edu/software/, http://nlp.cs.berkeley.edu/Main.html#Software) and have recently started using Scala as well (http://www.scalanlp.org/).
Scala's interface with Java collections IS a bit clunky. Thankfully, this is a library issue, not a language issue, and can be solved with better libraries. See http://github.com/jorgeortiz85/scala-javautils (for Scala 2.7.x) and http://github.com/scalaj/scalaj-collection (for Scala 2.8.x).
Really? For me one of Scala's best attributes, by far, is the ability to use its implicit conversions to make awkward Java APIs smooth like butter.
> One problem is that, based on the JVM, type erasure is a big "broken window" that seriously damages the Scala type system.
I'm not sure I understand or agree. Understand: If you don't use casting, Scala's type system seems just as strong as ML, albeit not as good at inference. (F# has the exact same inference weakness as soon as you start using OO and inheritance.) Agree: having the ability to consciously drop into a dynamically typed/runtime-casted world for small bits of code is quite nice (true for F# too). And I've yet to see a place where erasure (a) caused, even in theory, a runtime type error, or (b) lost any type information within the scala language at compile or link time (which is where the type checking lives). I think it's a red herring / mere implementation detail.
> As for oCaml, the real excitement is in F#. You get the wonderful oCaml language and all of its features, plus you get an interface to the C#/.NET world that's much nicer than Scala's interface to the JVM.
Counterpoint: I've written production code in both Scala and F#; give me Scala any day of the week. I really wish its .NET support was more mature; I'd ditch it in a heartbeat for Scala if I could.
To be more specific, when I want to transform a collection given by a Java library in a functional manner, I usually ended up with one or two localized import statements - to avoid e.g. scala.collection.mutable leaking into my other methods - and additional calls to convert the Java collection into a Scala specific and back again. This doesn't look like a big issue, but it increased the line count for a lot of cases from '1' to 3-5 ... which puts it uncomfortably close to 'dumb for loop territory'.
Admittedly, this is a small price to pay compared to the constant annoyance that is Java, but for my part I fear that there might be a slight tendency in Scala to fix interesting problems preferably to useful ones.
... lost any type information within the scala language at compile or link time...
How about this use case?
match { case x: List[Foo] => ... case x: List[Bar] => ... }
give me Scala any day of the week
Why do you have the preference for Scala? I'm curious because Scala is my language of choice for my personal projects, but I would ditch it the second a credible F# derivative would appear on the JVM (F#'s stronger emphasis on functional programming, cleaner syntax and abstaining from 'fixing typing' in OOP are the reasons for my preferences).
F# has basically two type systems -- the O'Caml/ML one, and C#'s. As a plus, you get full type inference so long as you stick with ML discriminated unions or records; as a minus, you lose the ability to apply concepts from OO without converting them to classes -- in which case you lose the ability to apply some handy ML-isms.
Scala tries to pull off a fairly deep unification of OO with functional types; for example making discriminated unions into case classes; or blending OO ideas of objects and classes with ML ideas about modules (including functors) into a single concept. It's far more ambitious in this regard than F#, but not necessarily an unqualified success either.
Co/Contravariance (maybe View Bounds?) in Scala enable you to encode this relationship, but from my understanding are somewhat handicapped by type erasure.
Another problem Scala tackles is that using inheritance as a 'code sharing facility' is a bit tricky (e.g. tractability and fragile base class problem) and not composable. Scala's traits and Self Types are a real improvement in this regard. That the order of mixing in traits can have an influence the objects behavior while not its type can be seen as a problem though.
F# takes the .Net object system 'as is' and introduces 'types' more aligned to functional programming (aka. discriminated unions) as separate entities. It does not try to integrate both concepts as does Scala with its case classes.
I think the other poster was right about this being more of a library issue than a language issue. In my experience it was rare to do more than wrap Java collections to or from an Iterable[]; that could be done easily without having to bring full mutable collections into the namespace.
Also, it might be part of the cost of working with "enterprise" Java, but it was actually really rare for me to work with a vanilla Java collection rather than some library's implementation of its own damn 'typesafe' iterator, like this: http://xerces.apache.org/xerces-j/apiDocs/org/w3c/dom/NodeLi.... It was super-nice to toss together my own wrappers of some library or another's semi-standard iteration API and have a ton of functionality come along via the magic of mixins; and get it all automatically applied for the cost of a single import statement via implicits.
> match { case x: List[Foo] => ... case x: List[Bar] => ... }
I suppose such a thing is made impossible via erasure, it's true. I haven't encountered a need for it -- maybe because the problem there is erasure + dynamic/runtime typing, rather than erasure on its own. Also it's something that's coming for Scala, it sounds like. (I haven't paid as much attention as I should to its ongoing development..)
> Why do you have the preference for Scala?
Part of it's just the intuitive feel: When I code in Scala, I'm writing in Scala -- it might be a fairly huge language, but it is its own thing. Coding in F# feels like bouncing back and forth between C# and O'Caml, depending on how functional you're feeling at the moment -- here's a C#-ish clause; there's an O'caml-esque one.
As an example, I don't believe you can use both inheritance and discriminated unions in F#. In Scala you can and it's actually quite useful.
Another example is that in F# you can omit type declarations on function arguments until you start using OO, then they become mandatory and infect your code. On the one hand the extra inference is nice but on the other it really drives home that you are coding in two languages, not one.
Besides that intuitive part, I think that Scala's module/class-level type system, although complicated, helps tremendously for writing big, complex, programs. Traits/mixins, flexibility in type constraints, and allowing types as members of other types add up to a really powerful ability to modularize without sacrificing. At least for me, a lot of my older O'Caml projects made fairly heavy use of functors. F# didn't bother to try to support the idea and only supports plain-vanilla C# interfaces. Scala embraced them, extended it, and made them much better.
Could you give me a simple example how say filtering and mapping a collection received by a java library and pushing it back to a method expecting a collection of the original type would look like? (quite likely that I overcomplicate this in my own code)
Assuming you want your maps and filters to be purely functional, you need only two simple methods: One that constructs an object that implements the Iterable[X] trait from the java object, and another that constructs an instance of the java collection from an Iterable[X]. All the implementation that you'd like is already done for you in the iterable trait, but you can always selectively override them if you'd like to provide a more tailored implementation.
Then it's up to your taste whether you'd like the conversion methods to be implicit (and thus available with a single import but making your code more 'magic') or explicit (single import + adding calls to wrap/unwrap methods).
Total overhead: centralized conversion methods, assuming you write them yourself = implementation of elements(), and in the iterator, next() and hasNext(). One import per module that uses the conversions, and optionally, explicit calls to perform the conversion.
Also, in Scala 2.7 I wrote my own library to wrap Java collections that didn't take much time or effort and allowed me to seamlessly pass scala collections to java code and use java collections the same as scala ones. Scala 2.8 is supposed to have such a library built in so you don't have to write it.
Lastly, your comparison between Scala's functional lists and Java's ArrayList is pointless. Scala has an ArrayBuffer class too, so you can choose between either as appropriate. You can also pattern match against ArrayBuffer as well.
Is Mono actually usable with F#? The last time I checked (2 months ago) the F# plugin for MonoDevelop didn't work and the questions on how to getting it to work (by various people) went unanswered by the developer.
The runtime performance of F# code was shaky at best, the REPL had (on OS X at least) startup times to make it practically useless and without TCO the usefulness of F# really took a hit.
Not wanting to bash on Mono or F# (which is my favorite language), but writing production software with this combination is currently not something I would recommend.
Performance and user experience on Mono are something we're definitely trying to improve - we spent a fair amount of time trying to make our Mono story better for our 2.0 release, and plan on investing more in this area as we move forward.
We can't be everywhere at once, so your feedback is really important here - If you're having problems with our tools on Mono, please let us know via Microsoft Connect or fsbugs@microsoft.com. Also, don't be bashful about filing Mono bugs. So far, they've been really great about responding to any issues that have cropped up.
Thanks!
Kind regards,
Vasili I. Galchin
On http://github.com/vasili/FSharpBinding is an FSharpBinding that works with MonoDevelop. Please read the README on this web site. I have tested new F# projects and several existing F# projects like Mandelbrot set, Stephen Wolfram's Rule 30, etc. This binding still needs polishing .... working on that with other colleagues.
Vasili I. Galchin
Here's a ridicuously detailed table comparing F#, ocaml, haskell, scala, ML (ie. everything in the article but scheme), at the aptly named http://hyperpolyglot.wikidot.com/ml
Is their work in applying ML theory, or creating more of it? ML almost always involves large datasets in practical use, but when you're trying to come up with a new method for ensuring correctness in some edge-case, you would be testing it with hypothetical extreme inputs, not feeding it mounds of data that won't trigger the problem. Haskell works well to model computer science research problems—that is, research into CS, not using CS to research other domains. Once your model has "solidified", you should by all means optimize it in C before running it on real-world data.
It has been my experience that it is difficult to get a bug passed the
Hindley-Milner type system.
It's type-checking all the way up.
I mistyped a single letter in an email and offended a coworker for life. He was on the road; I was in the office. I argued persuasively about a difference in design, ending with "When you get it, we can talk some more". I had intended to say "When you get in, we can talk some more".
Ugh. Have you considered...
I had some 'splainin to do!
What does it take to break into the general public's consciousness? Clearly it's not the parens.
I seriously think Clojure is too much to take in at the first introduction. It's really hard to get your brain around everything done right, when compared to traditional programming. You need to use it for a while to really get why it's approach to state (not concurrency) is awesome.
</contributor-bias>
Because really, Clojure is in many ways simpler than both Scala and Java. You can ignore the hard parts (agents, deftype, macros, de-chunking seqs, etc) and just cruise through the easy parts with a fair degree of ease. Everyone understands functions, after all. And Javascript has really pushed the "use functions for everything" approach into the public eye quite nicely.
http://mmcgrana.github.com/2010/03/clojure-web-development-r... http://github.com/cgrand/moustache
There isn't a mature web framework for Clojure yet (for obvious reasons), so there isn't a popular "way". At the moment you have a lot of good libraries and you can put your app together in whatever way seems most sound to you.
Clojure has a lot going for it in terms of theory but I don't know that it can claim any of the social proof of Scala.
The closest thing I can think of for Clojure is all of Siebel's Lispy interview questions from Coders at Work that lead an underinformed coder to think highly of STM and of Lisps.
Edit: Scala apparently predates Clojure by a good four years. The larger mindshare is no surprise in that light. Also it appears to have simple pattern matching and several other things that Alex Payne liked: http://www.slideshare.net/al3x/why-scala-presentation
In fact, a good friend of mine is working to put scala bindings over all the clojure data structures so they can be used transparently in scala. They're that good.
Wrapping the PersistentMaps took like less than 200 lines of code: http://github.com/codahale/yoink
From my point of view 200 LoC giving me something I already have is a lot. From the point of view of your friend 200 LoC might be a very small price to pay for a good learning experience.
I'm also slightly confused as to why you think reusing stable, proven, and reviewed data structure implementations is a "good learning experience" as opposed to being SOP.
In contrast, Clojure's PersistentHashMap dates back to 2007 and was in both the 1.0 and 1.1.0 releases.
In the meantime, I benchmarked Yoink's PeristentHashMap against 2.8.0.RC2's immutable.HashMap by a) creating a map of 100,000 sequential keys and then removing them all, and b) creating a map of 100,000 random keys and then removing 100,000 random keys. In the first case, Yoink was 20.7041% (±1.83693%) faster. In the second case, Yoink was 24.7834% (±4.17055%) faster. (Student's t distribution, 95% confidence interval, source is in src/test on the GitHub project and I used ministat for the statistical analysis [a cross-platform port of which is also available on my GitHub page].)
So to summarize, I have a provably faster hash array mapped trie at almost zero cost right now because I spent a few hours with a "good learning experience."
What I wanted to confer was that, you _might_ have been wasting a lot of effort because I assumed that you might not have been informed about work of the Scala team that has a high likelihood of obsoleting your project in the near future. Obviously you were informed and my notice was inappropriate. Be assured it won't happen again.
If increasing the performance by about 1/4 for your use cases - compared to the current beta - is valuable for you, great! For me it wouldn't have been worth a few hours. 'A good learning experience' would have. But obviously your values are different.
I just wanted to point out the (for me at least) non obvious fact that the mind behind the theory of Clojure's persistent datastructures is working on Scala's collection library and that Scala's hash maps are already backed by hash tries - http://www.scala-lang.org/archives/downloads/distrib/files/n...
And it's pretty neat to see a scala proponent call a scala project a "waste of time." Way to represent for your community, bro.
Personally, I have found haskell to be hard to program in initially, but once you get started, refactoring and feature changes feel "easy".
Erm... While Scheme can be useful for web and prototyping, it is typed. That's why R5RS requires functions like number->string and so on.
"What To Know Before Debating Type Systems" (http://www.pphsg.org/cdsmith/types.html) is a pretty good introduction to types.
Also, while I'm only partway through Benjamin Pierce's _Types and Programming Languages_, it's been quite good so far.
It points out in the "Static and Dynamic Types" section that what is meant by "type" in static type-systems and what is meant by "type" in dynamic type-systems are two very different concepts. From the perspective of someone interested in static "types", dynamically-typed languages don't have types at all.
So you have something in Scheme like:
(define foo 1)
(define bar "1")
(+ foo bar)
Contrast this with what I would consider an untyped language like Python or Perl, where the translation from string to number is done without any warnings showing up.
Very few languages lack a strong notion of run-time type. C and friends are the only ones I can think of where you can treat a block of memory as more than one type without the compiler or runtime throwing a fit.
http://www.norvig.com/java-lisp.html
Not clear how much is due to the functional nature of Common Lisp, but does show a clear difference between programming languages in expressiveness and productivity.
Haskell's type system literally allows the compiler to infer the correct code to run based on its type context
It literally has overloading! Imagine that!
Perhaps if you don't know Haskell, you should learn Haskell to understand what the article is saying.
Edit: This might help: http://www.haskell.org/tutorial/classes.html
I also believe that type inference is of limited practical application at scale. On a local level, internal to a module or function, it can make a lot of sense, but it's at risk of underspecification at the module interface level. For example, when you modify function bodies, you may inadvertently add more constraints to an inferred type (e.g. use an operator or function defined on a typeclass not previously brought in), and consequently break clients of the module.
OCaml (and potentially Haskell, though I haven't used this feature in Haskell) ameliorates this problem by giving the programmer the facility to specify (or generate and edit based on inference) module signatures that are then checked against. With this approach, I would argue that type inference actually makes your large-scale code more robust and easier to modify than without it due to the automatic verification of module signature. If you inadvertently introduce extra type constraints to your module's interface, the type-checker will tell you and save you from releasing a module that is incompatible with previous versions of the same module.
Without type inference, you're either annotating types everywhere (looking at you Java...) or waiting until runtime (or production!) to find out you've made an error (e.g. perl, python, ruby and on and on).
I really don't see how type inference and its associated program analysis and verification is anything but good.