Technology Change: .Net to Scala
randonom.com
randonom.com
One of the reasons why the migration worked for us was that one can (with a bit of discipline) get into idiomatic Scala slowly. You can definitely start off writing Java-ish Scala and still gain a lot from the decreased verbosity and nicer APIs. You can safely stay away from over-using implicits, functional programming etc. till you get a good grasp of the language. With excellent Java interoperability, you don't have to convert your entire stack to Scala overnight, but slowly replace old code with Scala.
With respect to performance, if you insist on writing idiomatic Scala, there is definitely a performance overhead, because the JVM is not suited for large amounts of small allocations, as is often the case with an immutable style of programming. However, we run a pretty large scale data crunching system, and depending on what your code does, CPU bound performance hit pales in comparison to the network and I/O waits, so for us, it was not a big deal. When we absolutely needed to optimize a tight loop or something, we wrote C-like Scala, using while loops and mutable data structures.
Addendum: The greatest win for us was the readability of the code base. Scala allows you to largely keep your data structures immutable, so it's much easier to reason about. With parallel collections, Actors and Futures, concurrency support is excellent as well. You definitely need to pick up a few new concepts, but the time is well spent as once you get a hang of those things, your productivity will be 10x - you don't have to wrangle with primitive concurrency constructs.
I do prefer Scala's collection types and for comprehensions over LINQ though. I find them much easier to read and more powerful.
For a real example, try designing generic data-structures in Java, that exposes the below functionality in a type-safe way:
scala> val list = List(9, 7, 5, 10, 11)
list: List[Int] = List(9, 7, 5, 10, 11)
scala> list.sum
res0: Int = 42
scala> list.sorted
res1: List[Int] = List(5, 7, 9, 10, 11)
scala> val stringList = List("a", "b", "c")
stringList: List[java.lang.String] = List(a, b, c)
scala> stringList.sum
<console>:9: error: could not find implicit value for
parameter num: Numeric[java.lang.String]
stringList.sum
^
scala> val bitSet = collection.immutable.BitSet(1,2,3,4)
bitSet: scala.collection.immutable.BitSet = BitSet(1, 2, 3, 4)
scala> bitSet.map(_ + 1) // the result is still a BitSet
res3: scala.collection.immutable.BitSet = BitSet(2, 3, 4, 5)
scala> bitSet.map(_.toString) // the result is not a BitSet
res4: scala.collection.immutable.Set[java.lang.String] = Set(1, 2, 3, 4)
scala> map.map(kv => (kv._1 + 1, kv._2 + 1)) // result is still a Map
res7: scala.collection.immutable.Map[Int,Int] = Map(2 -> 3, 4 -> 5)
scala> map.map(kv => kv._1) // result will not be a map
res8: scala.collection.immutable.Iterable[Int] = List(1, 3)
Ironically, Martin Odersy, creator of Scala, is one of the core people behind Java Generics [0]
EDIT: Just to clarify, I'm a huge Scala fan, and understand Odersky had constraints and compromises to do on Java, which I think is probably one of the reasons he went on to create Scala.
var list = new List<int>() { 9, 7, 5, 10, 11 };
list.Sum();
list.OrderBy(i => i); // list.Sort() is in-place, so using functional OrderBy instead
var stringList = new List<string>() { "a", "b", "c" };
stringList.Sum(); // C# signals a typing error
var bitSet = new HashSet<int>() { 1, 2, 3, 4 }; // not sure what a Scala BitSet is, but it looks a collection of ints?
bitSet.Select(b => b + 1);
bitSet.Select(b => b.ToString());
// map wasn't defined in the snippet. Assuming it's like a Dictionary<int, int>?
var map = new Dictionary<int, int>() { { 1, 2 }, { 3, 4 } };
map.ToDictionary(kv => kv.Key + 1, kv => kv.Value + 1);
map.Select(kv => kv.Key);http://blogs.msdn.com/b/dotnet/archive/2013/09/12/immutable-...
C# is mutable-first but well capable of providing immutability (requires good developers but so does Scala/F# :p).
C# is more capable and saner than Java and to tell you the truth, C# has been the first static language that I used with joy. If it weren't for Scala, I would have prefered C#. Skipping over platform limitations and considering just the language, I still prefer Scala any day of the week.
Let me outline the differences between the examples I've posted and the ones that you did.
list.sum
In C# or Java, the concept of adding numbers is not being exposed as a standard interface. In C# the + operator is a static member, so it cannot be part of an interface (in Scala there is no such thing as static members btw). In Java the + operator is something special reserved for primitives. In Scala it doesn't matter how + is exposed, because you can implement a Numeric[T] for whatever type you want and hence you can treat any type you want as a number in generic data-structures and algorithms. If you implement your own Complex type or RomanNumeral type, no problem, as you can sum them up as long as you also implement Numeric[Complex] and Numeric[RomanNumeral].
In contrast, for your sample to work, the C# engineers had to add extension method overloads of IEnumerable.Sum() for each number in the system. But what do you do if you want to add your own number implementation, like a BigDecimal or something with better performance for your use case. Yes, in Scala it works, without hacks.
list.sorted
var list = new List<object>() { 1, "2", 3 };
list.Sort()
There is value in doing this, rather than doing it with an OrderBy that takes a function for the comparison operation. First of all, many types have natural ordering and the inability to implement this means that you can't express natural ordering in your language. Also, these are really simple examples. You don't see the value of this until you start working with these tools in the real world in more and more complex problems.
Your examples with the HashSet and the Dictionary are wrong. They aren't doing the same thing.
In Scala, a BitSet is a Set of integers that stores those integers efficiently in-memory. It's a different implementation from a regular HashSet, specialized on storing integers. When mapping over a BitSet, you want the result to still be a BitSet, otherwise you lose its properties. But the result can't be a BitSet if you're not returning integers in that mapping function. You missed the point of my example.
So basically Scala allows you to override the returned type, based on the given mapping function and it does so in a statically type-safe way, as the real return type is noticed by the compiler (i.e. it's compile-time ad-hoc polymorphism, not runtime). And best of all, the mechanism is pluggable so you can always override the given defaults.
Of course, you can have dozens of utility methods that allow you to escape the limitations of your language. You can build anything on top of Java and C# actually.
It's still death by a thousand cuts and in case you can't see it, ask yourself why you haven't done functional programming in C#, even though it is possible to do so. And yes I know, some people do it, just like some people are doing OOP in C.
Has the situation changed totally? I don't know. Maybe Scala has improved significantly since then and all the pain points back then have been addressed nowadays. I'd like to be enlightened on these factors.
[1] https://news.ycombinator.com/item?id=3297750 [2] https://news.ycombinator.com/item?id=3292555
This is the thing I find most annoying regarding Scala use, and people advocating its use, it just doesn't do anything that Java can't do - plus the tooling for Java is wayyy better, there is a smaller learning curve, you can hire people to work with it and you don't spend your days endlessly debating what obscure language feature that may lead to tighter code in release 2.11.B.3.48d.M1.
Perhaps you would like to discuss the "academic" benefits with the likes of Twitter and Foursquare?
> "it just doesn't do anything that Java can't do"
It's a more expressive language. You could argue that Go and Python can't do anything that C or assembler can't do. That doesn't make Python irrelevant.
> "plus the tooling for Java is wayyy better"
In terms of the JVM the tooling is exactly the same. In terms of coding I use IntelliJ for both Scala and Java development, without noticing much difference in quality between the support for either language. In terms of libraries Scala has access to all the libraries that java has as well as additional libraries designed specifically for Scala.
Most (actually all) Java developers that I know grumble when having to code in Java after they are exposed to Scala.
Academic vs Practical probably? What's the point of discussing with <big-name-company>? Devs there do like to try out new tools, new languages, new platforms for the sake of trying as well. Devs are humans with ego as well, just like the rest of us.
> Most (actually all) Java developers that I know grumble when having to code in Java after they are exposed to Scala.
I code in Java, Python, JavaScript (modern) and playing with Ruby/Rails lately but I never grumble when I have to go back to Java because I know there are limitations/disadvantages of the other platforms as well. I use SVN and Git and I never grumble when I go back to SVN on a very large project with multiple component that have dependencies.
Context is everything.
YMMV.
Scala may be good for one-dev company; but terrible in terms of readability and maintenance for a team where more than 3 programmers work on the same code.
People new to a technology have a tendency to flounder around a bit in the beginning because they're naive to common coding standards and principles. After those are found or established then Scala codebases are just as easy to collaborate with on than any other.
[1] http://www.wired.com/wiredenterprise/2013/09/the-second-comi...
Let me address some of the points raised ...
(1) Scala is difficult for Java developers to learn, because Scala has new concepts in it, like type-classes or monads and these concepts are exposed within a static type system. Are these concepts useful? After using Scala for the last 2 years, I can bet my life on it. However there's a learning curve involved and Scala is definitely more expensive to learn than Java. That's why context matters. If a company has the resources to invest in this learning curve, then it's worth it, otherwise it isn't.
(2) Scala is not complex as Joda mentions. Not in the original sense of the word "complex". Scala is in fact pretty simple, as the features exposed are pretty well thought and orthogonal. But those features are powerful. The real complexity comes from its interactions with the JVM, as they had to do some compromises. It's important however to differentiate "easy" versus "simple". See this awesome talk by Rich Hickey ... http://www.infoq.com/presentations/Simple-Made-Easy
(3) As any new community, the Scala community started by doing lots of experiments, pushing its syntax and its type-system to its limits, going as far as to use features that aren't fully baked yet. This happens with languages that try new techniques. But as the community is maturing, those problematic edge cases get flushed out and there are fewer and fewer libraries that go nuts, as more and better best practices emerge.
The favorite example of Scala bashers is the Dispatcher library, that originally went nuts over operator overloading, but that was rewritten [1] and these days Scala libraries are actually quite sane and elegant.
Also, there's nothing wrong with the existence of experimental libraries, like Scalaz. Contrary to public opinion, it's not in wide usage in Scala projects, it's very useful for people that need it and such projects end up exposing weaknesses and pushing the language forward. The existence of libraries like Scalaz is a virtue and really, people work on whatever they God-damn please, you can't blame a whole community for it. Joda used it as an example for the dramatic effect, in a private email, OK?
(4) SBT's syntax looks awful until you get the hang of it, because it uses operator overloading to achieve a declarative syntax for building an immutable data-structure that describes the build process. This syntax will likely get fixed, but it's also a really pragmatic tool and I now use SBT even for Java projects and I miss it when working with Python/Ruby. There's also a Maven plugin as an alternative and Joda mentions the stalled Maven plugin and its lack of support for incremental compilation, however that's no longer true.
(5) Joda mentioned problems with the upgrade cycle. Scala 2.8 should have been in fact called Scala 3.0, as the changes in it were dramatic, which is why when Joda wrote that email, many companies were still on 2.7 and the upgrade to 2.9 scared the shit out of people. However, things are a lot better these days. Minor versions no longer introduce backwards compatibility issues, so if you're using 2.10.3, then you won't have any problems with libraries compiled for 2.10.x and you can even use many libraries compiled for 2.9. It's much more stable.
In regards to why is backwards compatibility affected by new versions, well it happens because the standard library hasn't stagnated. Java has awful pieces of junk in its standard library that were never pulled out or redesigned, but as a new and still growing language, Scala cannot afford to freeze its standard library. And we are not talking about major changes here, just a simple addition of a method to a base interface can trigger the necessity for recompilation.
But these days there are a lot of useful libraries with multiple contributers and that get compiled for newer Scala versions as soon as those versions come out. And personally I haven't bumped into problems because of it, as upgrades for the stack we've been using have been really smooth.
(6) At my old startup, we've built a web service that was able to handle 30,000 requests per second, with responses being processed in under 10ms on average, using only 10 EC2 h1.medium instances. Actually we started by running that code on 10 Heroku dynos, but then moved to AWS for more control. And the code was written largely in a functional style, using Scala's standard immutable data-structures all over the place. People that complain about performance degradation do have a point in the proper context, but people that bitch about it based on other people's opinions or based on silly benchmarks, well such people don't know what they are talking about, especially since immutable data-structures help a lot with multi-threading woes, giving you room to efficiently use the resources available.
Actually, I think Joda's actions of eliminating Scala's data-structures or closures from the whole codebase, were excessive (and I'm trying hard not to say silly).
(7) It's all about the libraries. If every library you're using is a Java library like Joda mentioned they did (as Scala makes usage of Java libraries easy), then you might as well use Java as a language. But if you're using Scala's strengths and libraries built with Scala, with a Scala-ish interface and you like those libraries (e.g. Play, Akka), then Scala suddenly becomes inexpensible.
(8) Stephen Colebourne, for all his talent, is a douche-bag.
[1] http://code.technically.us/post/54293186930/scala-in-2007-20...
This sounds similar to some of the stories about Twitter's switch to Scala. The developers who liked Ruby loved Scala, while the rest of the team stuck with Java, I believe.
Not true. But hey, its a free country. You can believe whatever you want.
Having C# background, I only took a look at Scala and I liked it at first glance, I wouldn't mind learning it some day.
See https://gist.github.com/samskivert/5278311
And although that's off-topic, while we're at it: http://www.digitalmars.com/d/archives/digitalmars/D/Java_str...
- Lack of a yield statement equivalent(in c#/f#). I know, there are probably a dozen alternatives such as continuation passing etc, but none so convenient.
- It's a massive language. Operator overloading seems to be highly encouraged, and library authors go to town with it. The Parser combinator stuff itself is enough to make one go dizzy, if you aren't familiar with it.
- Tools: Idea is OK, but not a patch on VS and F#. Compilation times are practically instant for small/medium projects in VS land - here it's glacial, even on a recent MBA. SBT is great, when it works.
On the + sides:
- If you know f#, you practically know Scala - well, enough to be instantly productive.
- Play 2 is very close to ASP.NET MVC. This was a welcome surprise, and a relief as well.
- It's such a relief that whatever you build will run on anything with a JVM. Xamarin/Mono is awesome, but those guys are still a little obscure, and a tougher sell than a JVM-based language.
Clojure/leiningen has the same issue (although there are ways to get the source itself as well [1], but nowhere as clean as rubygems).
[1] http://stackoverflow.com/questions/14395413/how-do-i-make-le...
In fact in Rubygems, you can point to a Ruby git repository (like github) and have dependency management happen from there - no necessity for a centralized repository (like maven.org). This is how you share a ruby gem - http://guides.rubygems.org/publishing/#sharing_source_code . I can now specify in my Gemfile to pull a particular branch/tag/version of the gem directly from revision control.
The same thing happens with SBT or leiningen, except that they are JARs. I believe this comes from using maven as the backend. Therefore, either you need to setup a local repository or publish to Sonatype or some other place. This is how you share your scala code - http://www.scala-sbt.org/0.13.1-SNAPSHOT/docs/Detailed-Topic... . Remember, publishing means packaging into a JAR. If you were too lazy to package your code into a jar and publish it ... well, then it's cumbersome to work directly from the repo.
I really, really wish scala/clojure would enable the ability to work directly with source as packages - the big, big reason why it's not is the maven backend.
http://maven.apache.org/plugin-developers/cookbook/attach-so...
Can't you use maven with your scala project?
Most open source Java lib/framework that publish their JARs to the central Maven repository also publish their Source code and JavaDoc JARs as well. This is where I applaud the Java OSS community.
Are you running in to a lot of projects that don't publish their sources?
Also, SBT can use use other SBT projects in github repos as depedencies: http://alvinalexander.com/scala/using-github-projects-scala-...
That's a feature of Bundler, not Rubygems.
> The same thing happens with SBT or leiningen, except that they are JARs. I believe this comes from using maven as the backend. Therefore, either you need to setup a local repository or publish to Sonatype or some other place.
I don't follow. I mean, you don't need to setup a local repository anymore than you need to setup a local Rubygems repository.
Checkout this anonymized and trimmed up `build.sbt` for the project I'm currently working on: https://gist.github.com/sam/7162604
It's got a number of JAR dependencies, followed by project references to projects on the local filesystem as well as Github, neither of which are JARs.
In fact, because I don't have much clue how to distribute my own libs on maven.org, I don't bother. If you want to use it, just drop the URI into your build.sbt and away you go.
Is this not what you're looking for?
If you want to create a JAR (say for deployment), and you're not working with a Play application, and you don't want to publish the JAR to a repository, that's cool too. Just run `package`, or use the sbt-assembly plugin. Just takes a couple seconds.
SBT is basically Rake + Bundler + RVM + Guard + RDoc all in one. I'm not aware of anything that those do that SBT can't. It's pretty easy to pick up as well. People complain about it, but I've personally invested a lot less time learning SBT than I have in those tools and feel like I'm probably as proficient at this point.
So, I guess it's cool that this kind of stuff is working now!
The reason my code is SBT 0.13.x specific is just because .sbt file syntax was robustified in 0.13.x to cover most cases you'd previously need a Build.scala for.
You can do the same thing easily in 0.12.x, it just requires you do it in a Scala class that extends sbt.Build instead. In fact, that's what my code was a few days ago. I only just ported it to the new SBT 0.13.x syntax since the old syntax continued to work. We'd been using 0.13.x for awhile, I just hadn't got around to porting the build from being spread across build.sbt and project/Build.scala to consolidating to just the build.sbt.
The docs for SBT are actually really thorough, but to me at least, a bit hard to get into. I blame the theme at http://scala-sbt.org and the fact that there's no obvious "from basics to mastery" guide. The website feels more like a manual. Going deeper than you might want pretty quickly, but before you've got the breadth of it down. If that makes any sense?
That said, if you end up having to work with SBT for your day-job, you end up picking everything up pretty quickly once you're forced to. It's a lot more consistent than anything on the Ruby side. You may have a few syntaxes to choose from to add a libraryDependency for example (single, Seq, special operator), but the mechanics are all the same and directly exposed to you. You're building a definition for the build, not actually performing work, so as long as that dependency sequence ends up with the graph it needs you'll get the result you need. There's not a lot of magic to it. Despite peoples complaints about the operators (which honestly, there's only two or three you'll run into regularly, and it's not black magic, just look them up or Command+B in IntelliJ to see what they do).
> If you know f#, you practically know Scala - well, enough to be instantly productive.
Sure, but actually you can say the same about knowing Java/C# and then moving to Scala. But in that case you're missing a whole lot of features that Scala offers that F# does not: typeclasses, traits, implicits, etc.
I feel the intersection of FP features between F# and Scala is much greater than C#/Java.
So Scala is mostly OO with some FP tackled on?
I think that's interesting but also a big indicator of why you see such a big split-brain situation in Scala. FPers program FP-Scala, OOers program OO-Scala, but neither is dominating. As far as I can tell, Martin Odersky programs balanced OO-FP, but few others do.
(but i found it hard enough to program in ocaml, which is similar to f#, because i was always torn between oo and fp parts, so maybe i am biased or just Too Dumb For Scala)
Yeah, and there are still some unsupported functions and properties in ASP.NET that keep you from being able to just drop an MVC project in. :(
This, the "every single library needs to provide some clever DSL", is my major complaint with Scala. It's not really Scala's fault either, Scala's core libraries don't have this problem. It's 3rd party developers with no restraint. So it doesn't have to be this way and I think (or hope, based on some comments about Scala on this site previously) that it's beginning to become a discouraged practice.
Still, I have an immediate, visceral reaction to it after the years I spent trying to untangle heaps of overly clever C++ code with the same kinds of issues. I never want to be in that situation again.
Code should be, above all, easy to read. If your code looks like a different language depending on which library you're using, that's a real negative.
At least we now have pretty good IDEs that help out, but even then I think that it's probably not be a good sign if you kind of need to have a good IDE to understand the code.
Also, there are more than 11 times as many Ruby projects at GitHub as there are C# projects[3] (even though .NET developers vastly outnumber Ruby developers).
[1] https://www.ruby-toolbox.com/
[1]: http://www.nuget.org/ [2]: http://en.wikipedia.org/wiki/Ruby_%28programming_language%29 [3]: http://en.wikipedia.org/wiki/.NET_Framework
I remain a fan of F# on Mono. Love the FP paradigm, love all the easily accessible libraries, and love that it's all free.
Scala would probably be a close second choice for a development shop, though. I personally would move towards pure FP with Haskell or Erlang, but staffing becomes too much of an issue to scale that out. I don't think they have that issue with Scala.
Does it? We've hired several dozen Haskell developers at this point, and we always get far more applicants than available positions.
I try to stay ahead of the game on these things. If I'm still collecting data, either I'm missing some major sources or the overall market isn't ripe for it yet.
[1] http://www.indeed.com/jobtrends?q=scala%2C+groovy%2C+clojure...)
http://www.indeed.com/jobanalytics/jobtrends?q=scala%2C+groo...
edit: yeah, it isn't intended to be programming specific: "Indeed.com searches millions of jobs from thousands of job sites. This job trends graph shows the percentage of jobs we find that contain your search terms."
Under the graphics you have the links to the respective jobs. Just check for yourself what these "Go" jobs [1] and the "Scala" jobs [2] are. "Haskell" jobs are also interesting.
And more: don't forget that currently even Php ads feature the infamous "experience with functional programming (Lisp, Clojure, Haskell) is a plus" line.
The Microsoft build ecology is a little bit more mixed bag, but when compared to the explosion of choices for Java between Ant/Maven/Ivy etc. I'm not sure which I prefer.
As far as languages, working in C# makes working in Java so painful. C# makes so many better language/library decisions. The problem is that the JVM is a very compelling target platform (much more so than being stuck on Windows).
With those constraints Scala is a very nice language to use. In fact I dread every minute I have to spend in Java. Further everyone I know personally who has spent time in Scala feels the same way. I've literally never met anyone in person who prefers Java to either C# or Scala from a language point of view.
What other Scala tools you're referring to? Scala can reuse almost all popular Java tools.
Also, can you elaborate on strict coding convention? do you mean operator overloading?
The article has just been edited with the above note.
I think the lead by Groovy from 2009 to last year should be ignored because job ads for Groovy programmers want them to use it as well as Java, but those for Scala programmers want them to use it instead of Java.
I've gone through this process (tech stack shift), both as a recipient-of-change as well as a catalyst-agent-of-change. Remarking on the email that the OP's CTO sent to everyone, these statements stick out:
"...A technology choice will be made appropriately for the needs as we see them..."
"...no hard decision has been made about our future languages or technologies..."
"...Less C#/.Net, more Scala/Java/Open Source..."
Having been the guy in that chair, this is not an easy thing to do. Being handcuffed by prior technology decisions is not only difficult, it's often untenable. Usually, the people hired into those leadership roles often have to resolve the decision making issues that got the company to that point. That said, here's what I've learned in terms of how to go about these types of long-term changes:
- As the CTO, take full responsibility for introducing change. This is not taking credit, but instead being the owner of the decision and ensuring it succeeds. Support the team, and encourage them to openly question the decision, so that those questions can be addressed. The CTO's email above makes a change decision requisite, but it's up to everyone else to make it work. To anyone who's worked in a potentially charged environment, that's a typical setup for scapegoating. As the lead dog, don't put your team in that position. If your decision comes off as playing-the-executive card, you have failed.
- As a team member, embrace change but make it completely objective. Be open to change, but only as it helps to reach a goal. Evaluate technologies after using them, comparing them, and finding what's good and what's not-so-good. Make your personal productivity the least-influential point of analysis. If you can't find a way to assess something honestly because of your preference for operation, you're doing yourself a disservice. Be productively critical of technical decisions -- your own, and others.
The Technical Strategy email was sent out about midway through our adoption of Scala. I didn't explain the dynamics of our larger development department much in my post, but at that time we were essentially two teams in two different physical locations. The location I was at had already seen success with a few projects in Scala (BeetleJuice and the beginning of our API). Our CTO thought it was time to set a clear direction for the future of development department after an analysis of our current situation and discussion with members of the wider team.
Nobody was shown with the door. I'll admit there was some resentment, but nothing a well adjusted developer couldn't get over.
As I mentioned at the end of the post, one thing I would do differently in the future is make a stronger effort around consensus building about the decision. However, the Scala change started off as something small and organically grew into a legitimate alternative to the Microsoft stack. Going into our first project (BeetleJuice) we didn't anticipate the impact it would ultimately have on the organization.
My primary criticism of the CTO's email was that it didn't seem direct, which I've found is the most important aspect when communicating in that role. With all the intentions of being inclusive and providing avenues for flexibility, I've found the interpretations are most often viewed as ambiguous. The higher up the chain, clarity becomes so much more important. Live and learn, in my case.
In terms of consensus building, I had to do the same thing. For me, we accomplished that through objectivity -- we were constantly evaluating. We asked team members who liked technology A to provide brown-bag sessions on technology B. We had "skin-the-cat" sessions, where we would have a team of two or three implement a somewhat basic solution in three different ways (you know, nine ways to skin a cat....). It took some doing, but essentially what we did was enable developers who might not be so flexible to considering alternative technologies to get comfortable with those by spending time using them and providing an opportunity for them to say "this sucks" or "this is awesome". The team, as it was constructed, really responded well to this. Not only did they feel involved, but they were also tacitly preparing themselves to accommodate a shift, no matter which way it went. I was really surprised at how well the team became empowered; it was not something I was expecting to see.
Your CTO definitely had a tricky process and conversion to balance. I empathize, it's so easy to do a mediocre job with things that one wouldn't expect to matter quite so much.
Company heavily invested in X. New CTO comes on board, gives the technology stack a "hard look" which is often a euphemism of, "I wouldn't have chosen that technology, so you must change." Large portion of the existing developers leave, new developers (including OP) come in. New developers base technology decisions on what's best for their career, not necessarily what's in the best interests of the company.
After all, in the Why Scala section they plainly state they knew little of Scala and there's a large section about what languages are hiring.
And as you said, if there is time spent on what made them decide to switch away from the Microsoft stack, it is buried to the point I couldn't find it in two read-throughs of the very long article.
But, hey, the world still needs Cobol programers, so...
With all due respect, you obviously know nothing about the Microsoft stack. These people were not using VB6. How do I know? From the article:
new projects chose a base technology stack of C#, ASP.NET MVC
ASP.NET MVC is not an "old-school" framework. It was first released in 2009!! That's four years newer than Ruby on Rails. It's ridiculously actively developed as well as being open source. It's currently in version 5 which was released October 17, 2013 which was... OMG 8 days ago.
I continue to be shocked at how little most HN developers know about the Microsoft stack and how much innovative work has been done since they last looked at it in 1998.
(When we moved over to a Python-based solution, we ported from MVC 3/SqlLinq/MS SQL Sever stack)
EDIT: Oh, and were had a lot of good stuff going on using ServiceStack....loved that set of libraries.
My condolences...
Chasing shiny things is great if that's your goal. But if your goal is to build great product, then focusing on that should be the goal. There's still a lot of great product written in C, Obj-C, C++, and Lisp -- all rather "dull" languages compared to the shiny new stuff.
Against that, we weren't given a reason for the change. Maybe he was being nice and not exposing how things were. I'm certainly aware of dysfunctional teams that really need everything fixed, and there is little to do but wield the ax and get it done. But it is a heck of a gamble. These "old timers" - perhaps they were cranking out pretty good code at a good rate, on a stable system they understood, on time and budget, and so on, and had somebody come in, tell them they aren't buzz word compliant, and enforce things on them that neither solve problems that they have nor really offers much of an advantage. Or, perhaps they were dinosaurs that took 3 months of arguments between 5 committees to decide whether to label a button "reply" or "send". I dunno.
I've seen it both ways. I had a programmer react to my use of python as "oh, that silly little language". Everything must be written in C and reinvented by him. Not a very useful attitude. OTOH, if I was to storm around and argue that our existing, well working c code should be converted to Scala for no particular reason, I'm being the unhelpful one.
tl;dr: I didn't see anything in the article that made me think the existing team was doing it 'wrong' to incur such risk and costs. I don't much care if they are successful - the risk stands out like a sore thumb. We eviscerate the bankers that bet the company for their quarterly bonus by pursuing high risk, high reward strategies, and say "cool" when somebody throws out their team, source code, and institutional knowledge in favor of a cool new language and buzzwordy development process.
I highly doubt they're just throwing all their products away.
Irrespective of the merits of the technology [1], if hiring a new CTO results in a lot of the existing development staff leaving then something is very wrong.
[1] I work on the .net stack, but I've tried Scala and think its a great language.
.NET culture is not F/OSS culture. Different personalities and types of people in those camps. Different expectations of capability. When you are looking to hire non-Blub programmers, it's a lot easier to find it in F/OSS-land.
F/OSS tooling has a big priority on interop with each other. MS tooling has a big priority on interop with MS (and companies that MS has contracts with or has strategic EEE priorities on). Having a broader base of tools to use is a big deal.
F/OSS development progress can be monitored and input can be given (and patches can be sent in). This is in contrast to the proprietary approach where you pretty much have to take what you're given, and input is limited (depending on company). So it's easier to manage church and breakage risk in F/OSS.
Full-size enterprise MS licenses are not cheap. The numbers I've heard thrown around are big. And it seems that MS tooling is focused on vertical scaling, so your HW costs start looking very large as you stack your enterprise memory & HA storage systems into the box. This is opposed to "el cheapo" redundant pizzaboxes. So you start incurring a particular kind of expensive cost.
All of these are reasons I think F/OSS development is a better place to be, to hire from, and to want to go to. None of those reasons were given in the article, so we are left with speculation - was this just the CTO throwing his weight around? was this an attempt to move to a more mentally flexible organization? was this an attempt to cut costs?
Looks like many agree with you, so I'm really curious about what problems arise long-term from decisions like that.
I mean...our legacy code is in BASIC...but if they told me I needed to maintain the BASIC programs, I would probably quit.
When our CTO came onboard I meant to say that he gave a hard look at our development process, not so much the technology stack itself. I agree it's confusing since the post is about Technology Change and I'm probably confusing the point further by mentioning both the earlier process transition when our CTO came onboard and then the later Scala technology change.
Thanks for your feedback.
EDIT: I've updated my post to elaborate on why a new CTO was brought into the organization.
In .NET, MS provides everything, and the OSS counterparts are second-fiddle to the blessed libs. This is a problem since it stifles the OSS community trying to compete with first-party product... and while MS has generally competent coders and products, sometimes they make mistakes that are agonizing pain-points... and the MS obsession with "no breaking changes" means that those pain-points will never be fixed.
No MS mistake is ever fixed. It is replaced with a new library with a whole new week-long learning-curve and its own foibles and mistakes, which will, in turn, never be fixed.
On the other hand, creating a new ASP.NET MVC project with the official Microsoft template that ships with Visual Studio automatically pulls in open source libraries/frameworks like jQuery, Bootstrap, and Json.NET, not Microsoft-reinvented versions of those things.
On the other hand, we miss a lot of the churn caused by people chasing after the latest js library, which also seems to be a big preoccupation of the FOSS community. If you're building software that has to last for years then stability of the underlying platform and having fewer ways to do something can be an advantage.
Don't make me figure out how many database access approaches there are in stock .NET. :-)
More seriously, JS devs seem to have a thing for reinventing each others wheels - not just once, but tens of times (hyberbole). I really don't see that behavior in other communities by and large. The worst I've seen elsewhere is 3 similar Python libs
I dig appreciate the stability argument though. Which is why I'm a Common Lisp fan; its a standardized and mostly extensible language.
In the beginning there wasn't an initiative to migrate away from .NET. Management encouraged us to try new technologies all time and the BeetleJuice spike seemed to me like a good opportunity to try something new given its simplistic requirements.
From an organizational perspective, there was some discord about how expensive our Microsoft infrastructure was and how poorly it scaled. I won't blame Microsoft for all our scaling problems, as a lot of it had to do with our legacy architecture, but licensing fees were a legitimate concern. Management didn't have any clearly defined goals to move away from Microsoft, but when we started using Scala, Mongo, and other open source projects they took the opportunity to make the case for more migration of our infrastructure. This new direction was officially announced as part of our CTO's Technical Strategy message to the dev department (included in my post).
All they've done was get rid of smart people, create complexity, and embraced a culture of buzzwords.
A complete disaster in the making.
And for what?
Thanks for the feedback.
Though maybe that's what was required to get a buyout or merger, in which case that's what you needed to do due to the environment/culture of the other party...
http://www.empathica.com/press-release/mindshare-technologie...
The Mindshare acquisition is a recent event that postdates our technology change (Sept 19).
It wasn't too difficult to convince the CTO that Scala was the right choice. He's a big OSS advocate. From a technological perspective we were trusted to make the right decision.
I think what really sold the the rest of the management team was how much money we could save in licensing fees.
Come on, ...Microsoft licensing costs are a rounding error compared to the cost of developers.
Scala enjoys all the benefits of JVM ecosystem, which is arguably more familiar, predictable and popular than C#/.NET. That and the incredibly rich selection of OSS libraries.
Just kidding.
(I speak as someone who moved some web apps off a .NET stack to OSS, but that was for a small company's single product)
This does not make any sense whatsoever since e.g. ASP.NET MVC is OSS. Why don't you say ".NET stack to JVM stack"? Because it doesn't make your choice sound intrinsically better?
in general, when people mention ".NET STACK", they're talking about the whole-shebang, not a single library that might happen to be OSS when the other 100 libraries you're dependent on aren't.
ASP.NET MVC is an open source framework, sure, but the .NET platform, as such, is not OSS. (Some parts of the MS .NET ecosystem are open source, and there are open source alternative implementations for some parts, but that doesn't make the platform OSS.)