How We Moved From Microsoft ASP.NET to Rails and NoSQL
infoq.com
Tekpub.com was originally written using ASP.NET MVC, Microsoft's newest framework - which the founders new really well. However they recently moved to Rails and NoSQL.
infoq.com
Rob Conery is one of the names that any fan of ASP.NET MVC knew. He was involved in the project when he worked there (I'd call him the most high profile MS employee aside from Scott Guthrie). To have him come out and say he couldn't make a startup work with Microsoft technology really tells you all you need to know about that.
I still need to use .Net at work but I think this combines with a bunch of other factors to push me away from Microsoft on anything else I do.
I can see how it might come off like I'm damning the platform - that was hardly my intention. Rails has had 3+ years of Open Source contribution to build out the platform and make it do amazing things - ASP.NET MVC will (hopefully) grow in some of the same ways - but there's a somewhat stunted element to the .NET OSS ecosystem. Hopefully that will change.
It's also very slow moving and wildly outdated.
But you still had to switch and it still cost you time and money you wouldn't need to spend had you just gone with Flash in the first place. So the point to me is still "don't use Microsoft technology because you'll end up needing to switch"
C# is a fine language, and ASP.Net MVC is a great framework, but I can't help but feel like the 'other kids' are ransacking the candy store while I'm out on the curb. In terms of testing frameworks and tools, the Ruby/Rails platform is lightyears ahead. I know there are attempts to bring some of the core concepts in RSpec and Cucumber to the .Net platform, but we're not there yet, and probably never will be. The goalposts are always shifting - the OSS platform can respond to change and new ideas much more quickly than the MS platform.
We have an MVC framework for web development now, but it has taken 10 years (Java Struts was released roughly around the same time as Webforms iirc)! Testing is only one element of this - in the OSS world we're also seeing an explosion of asynchronous queuing/messaging frameworks, object/document datastores, serverside js and other innovations.
I also share your frustration with the coupling of the framework releases to a new IDE. I use Vim for everything other than .Net development, and like the fact that the rug isn't going to be pulled out from under me every two years.
As a Microsoft developer it is hard not to feel envious of the OSS community. The Alt.Net community if anything appears to be shrinking, and it is presumably because those developers have come to a similar conclusion as yourself - the cost simply does not justify what the MS platform can offer.
Right now I'm the CTO for a largish image licensing shop (think Getty, but more boutique/high end) that is entirely .NET (I inherited it). We are now in the process of moving it off Windows onto an OSS stack. It's a miserable experience. And an expensive one.
For starters, the thing is architected poorly, so that does play a big factor in it. I know, I know, it's always the new guy inheriting an existing system that lays to claim that the inherited thing is a piece of garbage, but in this case it is undeniably true. And I can tell you, as someone who has been doing .NET consulting for the last 8+ years, it's not uncommon for MS based solutions to be architected this poorly, but I'll get to the reasoning behind that later.
So yes, poor architecture, lazy design choices (the wrong kind of lazy) but what is most aggravating are two other factors: cost and maintenance. Of course the very first thing I did when I settled in as the new CTO was to figure out how the thing was maintained and how much the thing was costing us a day to run. At my former CTO gig, massify.com, one of my proudest achievements was getting the daily cost of running the site below the price of a pack of cigarettes. Now we are talking about a user generated video content site with not exactly insignificant amounts of traffic and some very healthy storage requirements. When we started out, it was running us around $300 to $400 a day, but by the time I left, it was about $8 bucks give or take. And during the three years I was there, the site never came down unless we took it down for some kind of database migration or similar updating. And all of this happens thanks to OSS software and the ease of use of unix.
Now we are in a situation where we are paying for the server OS's, the SQL server instances and a variety of other utilities to keep things running. Not to mention the yearly MSDN subscriptions. That shit is expensive.
But what is worse is that it's such a pain in the ass to update and work on. Remote Desktop is retarded, but I have no choice as command line maintenance is simply gag inducing. Something that would take me mere minutes in a linux command line takes me an hour or more clicking around like an idiot in remote desktop. Good luck remotely admin'ing a windows server taking a massive CPU load. Now when I was at Massify, I could maintain and monitor our whole fleet of servers from my phone - and on a couple of occasions even did so while sitting through the previews at a movie theater. Do that on MS? MMM ... no.
But back to bad architecture and why it's endemic in the MS development world. ASP.NET. ASP.NET is such a convoluted, poorly conceived, poorly executed pile of shit. THIS IS COMING FROM SOMEONE THAT USED TO SING ITS PRAISES. And when ASP.NET is a cornerstone of .NET development, I don't think it's implausible to rationalize the rampant bad development practices of your typical MS developer back to it and other MS technologies similar to it (I'm looking at you Sharepoint). You see, MS is chock full of shitty architecture. And MS devs learn from MS. It's a cycle of shit that is constantly perpetuated by each release of some new MS technology. And I can tell you, I've seen it time and time again - working for companies like Pfizer, R/GA, some of the top 5 interactive ad agencies, etc. Finding a great MS developer is like finding a great PHP developer. They exist, but are such a minority as to be almost mythical. We used to joke that MCSD stood for Microsoft Certified Shitty Developer. Well, we always pitched it as a joke anyways, in case anyone ever overheard.
So, now we are currently wiping the slate clean and moving the whole thing to an OSS stack. And so far, even in the rough beta stages, we've already demonstrated how superior that whole ecosystem is to where it was before. It's very satisfying to show our licensing agents our new shiny search system (using a heavily modified Solr) and watching their eyes light up as we do things with it that the old MS based search engine simply could never do.
So that's my rant about that. MS is dying shit as far as I'm concerned, only used by people who are somehow locked into it, or are the bad kind of lazy and don't want to challenge themselves. I know that's not very objective, but it's the lessons I've learned.
PS. When I was interviewing with the company, I literally had a 4 hour conversation with the outgoing CTO about the benefits of MVC. He couldn't see them and I couldn't explain it to him in a way he understood. He kept pointing at ASP.NET user controls and how great they were. Of course, several months later I find a page on the admin site that is roughly 26K lines of code in the codebehind with a nice spaghetti mixture of HTML and SQL and C# code. I nearly quit when I saw it.
The search engine is a customized version of Solr.
The front facing website is done in PHP using a framework we developed at Massify (http://heavyfuckingmetal.com/). We thought about going the ruby or django route, but the front end is so light functionally we decided to go with what we knew inside and out. We'll also be using MongoDB for certain things.
The way we are phasing the rebuild, the SQL Server is the last thing we're migrating, so that will stay in the mix for awhile. It's the most complicated piece because it has a bazillion stored procs that need to be migrated/refactored out. The structure is fine for the most part, though a little de-normalization is probably in order. Eventually I hope to have that running on PostgreSQL by next winter.
The backend, or admin, will be done with heavymetal and cappuccino for the front-end.
As an aside, the entire rest of the company is mac based, so we've written a couple of desktop apps, as well as some server apps, in objective-c/cocoa using quite a bit of open source libs. We'll be putting some of our own code public on github in the next few weeks. We've also done an iPad app for showing our image portfolio to clients.
Do you have an idea why this is the case? Or the other way around why the architecture of OSS projects seems to be saner?
disclaimer: I have no exposure to the MS stack, with the exception of playing around with F# which in my opinion is a very well designed language.
At some point, MS probably realized that either these tools are not enough or their customers are asking for more because businesses are getting more complex and agility is required.
MS, known as the company that is great in terms of partnering and pleasing everybody except developers, decided to re-brand its offering to be more developer friendly. But at the same time, still serving the non-developers community as well (which seems to be dying slowly because well... if you don't like computers, you will not like it at all, period. You will hire someone to do the "dirty" job for you).
It'll take a while to transform the old MS to the new MS. If they ever made it.
Meanwhile, (some) OSS projects didn't have ambitious goals. They solve the authors's problems. During the inception of the project, they didn't care about the non-developer audiences. They care about themselves: developers.
I only catch glimpses of the MS world when I have to use common libraries originally developed by our Windows experts, whose design decisions often leave me puzzled. I am curious about what you think about the idea, that OSS culture incentivized creating architectures conforming to fashion trends, while MS culture incentivized to finish feature X until Y - no matter what it takes (just do it Scotty).
e.g. in the OSS world beginning of 2000 J2EE spawned heavily over-engineered solutions, which were displaced by the convention over configuration trend exploding (mostly due to Rails) around 2006. And now the TDD/BDD crowd are fighting it out with the Architecture & Patterns gang with the DDD and DSL tribes assimilating the surviving concepts...
A member of a currently hip fashion trend will have a better staying power in a nasty flamewar caused by a random design decision about the color of a bike shed in an OSS project than a lone warrior. So you start having design styles competing for followers.
Meanwhile in the MS world (wild guess following) heavily funded 'central planning solutions' (from frameworks to design styles taught for certification purposes) offered by MS stymied competition of alternative approaches. A focus on - if I understood your correctly - usually very risk adverse business needs, while keeping the actual code of produced solutions hidden from outsiders additionally put developers under pressure to avoid experiments.
Would it be reasonable to say that we have an evolutionary system of small actors competing for developers interests (OSS) fueled by <insert book about OSS phenomenon> vs. a central planning approach of a major corporation backed by a huge war chest?
Good MS developers should learn to take what's best from the open source world and apply whatever is most useful behind the corporate firewall.
Let’s be honest about technology for a second: Things go wrong. No matter what solution you use things go wrong and for the most part non-technical management doesn’t understand that. When something goes wrong they want someone to blame.
If you decide to go with something like Ruby on Rails you have to contend with the fact that management doesn’t understand open source (or trust it) and have never heard of 37Signals. So the blame falls to you for having gone with a small “experimental” company rather than a “trusted” one like Microsoft (and yes those words in quotes are actual quotes from my attempt at using such technology)
Where as when something goes wrong with Microsoft technology it’s written of as inevitable (that’s where the saying “no one ever got fired for using Microsoft” came from) It’s a sad reality but sometimes you can’t pick the best technical solutions. Sometimes the extra time it takes to use proprietary Microsoft technology is worth the political capital you save by going with Microsoft. That fact doesn’t make me happy but it’s true.
But at the team and individual developer levels I think there's lots of room for good open source solutions, and this option isn't always considered because developers are looking to MS for the next big thing.
I believe this is reflected in the certification programs too, because they don't focus as much on building a sound programming foundation as on making good Microsoft evangelists. From Microsoft's viewpoint, why not? But it's that divergence again from the company's perspective.
You're very right, however in that no one gets fired for using Microsoft - but that's changing. Don't take my word for it - ask the guys in Building 42 - they know it too.
You touch on a very particular, hot topic in the IT world - that having a phone number to Microsoft will solve problems. Sometimes it does - but most often that comes at more than a bit of a price :).
On the flip side - my support channel is Google and I find my answers in about half the time as I would with a topic such as ASP.NET MVC. Rails is quite mature and people have tried everything with it - found where it's great and where it blows. I've had more to read on it than you can imagine.
Anyway - it is a sad truth and one that's about to change as the old guard at your company cycles out and are replace by people with knowledge and pedigree - namely people like me - who will systematically cull the policies that don't make business sense, and keep the ones that do :).
This doesn't mean "ax all things Microsoft" - it means "find the best value and leverage it". MS is changing its focus as it sees this - it knows this shift is coming - it's why ASP.NET MVC is Open Source and SQL Express is free, and more is on the way :).
On blame, every executive staff of a large company that I've ever seen casts blame. Yes there are startups with a small number of people that are all pretty even keel but as soon as you get to a certain size you'll get opposing interests and that's where blame comes from. The blame game is a political game played by people trying to get leverage over other people in order to get more resources. It's stupid but it's the reality of the world. If you don't see it you just aren't high up enough to be the one fighting for resources. Because as someone who read about Microsoft as a kid I know it was happening at Microsoft (Silverberg/Alchin?)
As far as brand, that's just human nature. Look at the side of any Grocery Store brand and you'll see the same ingredients as the big brands. But any grocery store manager will tell you Captain Crunch outsells generic white box crunch 100-to-1. That's why companies build up brands in the first place. That's why Microsoft brags about being one of the most trusted brands in the world.
So this really isn't old guard as much as it's the stupidity of human nature and I really don't see it changing that much.
P.S. This is Username: TomOfTTB not Username: SamAtt. If he doesn't want me posting as him he should learn to @#%& log out.
But you can always pick a better employer.
That doesn't make it right.
Not to say he's not a big name, but to say he's the most high profile MS employee beside ScottGu seems absurd. Even in the dev tools space I think you have to give that to Anders.
And easily the top .NET employee is Anders by far.
From your list of Technical Fellows it sounds like you're probably not a web dev, so you wouldn't have run into his stuff. An MS web dev these days would tend to list names like ScottGu, Haack, Hanselman, Conery, Walther, le Roy, Wilson, Verissimo, Lipton.
In other news, Anders is totally the cat's pajamas.
I reported my own experience moving from Asp.net to Ruby on Rails in my blog a few days ago:
"How an experienced developer multiplied his productivity tenfold" http://aymeric.gaurat.net/index.php/2010/how-an-experienced-...
If you want to test the world beyond MS I would recommend (apart from Rails or Django) looking into the Scala/Lift (my preferred toolset), Clojure/Compojure or JavaScript/Node.js combos (depending upon you language preferences).
I would be careful with Rails, when you plan to do some heavy data processing and/or if you need your application processes to run for a long time without being restarted.
For small, straightforward applications or prototyping the Ruby/Sinatra/MongoDB combo is also really really nice.
Heavy data processing yes, but the long running process is bogus. The major memory leaks in Rails were patched up years ago. The tooling is a lot better today on the off chance that you do need to debug a memory leak, and the new breed of ruby runtimes such as Rubinius will keep memory in even tighter control.
Even with the better runtimes now available, suggesting that they come close to the stability (and performance and long term memory consumption and profiling + debugging capabilities) you'll find on the big JVMs is not really helpful (JRuby being the obvious exception but even here I am sceptic).
I don't know ASP.NET MVC at all, so I haven't tailored the presentation to Microsoft developers, and I'd be curious to hear if you find my approach accessible. You can leave me feedback at http://www.railstutorial.org/feedback.
N.B. Although NoSQL is outside the scope of an introductory tutorial, I'm hoping to do some follow-on articles on such specialized topics once the core tutorial is finished. (I'm currently writing the final chapter.)
I saw a commenter talk about Linq-to-SQL and yes this is possible too. I use a PostgreSQL server and use Linq-to-SQL with it. It works but as expected, the documentation is poor.
This is free to use, but from my experience some of the costs can be very frustrating time spent trying to figure out why things are not working as they should, or barrier to entry with poor documentation.
As one commenter pointed out, mono behaves strangely when changing part of your codebase while it's running. Modifying a file can cause 500s or timeouts to be thrown. Killing and restarting the fastcgi process is needed.
A major positive is the developers who work on these parts of mono are very responsive to issues and code is improving all the time.
From the article:
"JA: All the problems we had (licensing, testing, deployments) could have been overcome. We could have used workarounds, written our own deployment framework, etc. What is comes down to is that we both really enjoy working with Rails and we enjoy the Rails community and the tools and libraries available. One of the best parts about running the show is that we get to do what makes us happy."
RC: Money.
1) Continue to use a platform that costs more and isn't as much fun.
or
2) Use a platform we both love using and works out to be cheaper.
Pretty simple decision really.
I'd also be very hesitant to use MongoDB as a main datastore as it has no durability model - the 'D' in ACID. I hope they've at least changed the setting, or don't mind losing their last 60 seconds of writes when their EC2 instance goes down.
http://www.mongodb.org/display/DOCS/Durability+and+Repair#Du...
Note that Mongo compares itself to MyISAM, but that is infrequently used now because of its lack of durability amongst other things - no transaction support, etc (most apps use InnoDB).
I think they made the point in the article that they could have overcome the obstacles of the ASP MVC platform, but why? They said they're more productive on Rails, they like working on it better, and it's cheaper.
What positive would there be (for them) of staying on the MS platform?
Visual Studio is $1200 a pop - this is for two people.
I'm already at 5 figures and I didn't include Expression/Media Encoder and so on.
http://www.microsoft.com/sqlserver/2008/en/us/pricing.aspx http://www.microsoftstore.com/s/visualstudio
1) It's expensive compared to other hosting platforms. 2) I hate lock-in to a particular platform. If we coded against Azure tables we would be locked in to Azure. If we used their SQL platform we would be space constrained. 3) I have used Azure and deployment, updates, and troubleshooting are all slow and cumbersome. It is still a very immature platform.
We've found that like for like the Azure platform is very well priced and is actually less expensive than our current Ubuntu setup, as well as being cheaper than comparable high level Microsoft hosting providers.
We don't much like lock in either and for the parts of the Azure platform that will lock us in we're extracting the code into a wrapper so we can replace with the minimum of work should we need to switch hosting platform in a hurry. Database wise we're going with SQL Azure and their 10GB limit I find to be quite spacious for our needs, although that would depend entirely on what you're storing. I believe they have 50GB databases in the works for the near future?
The deployment times are an issue and I have heard similar, but again I believe if you upload the project yourself then request the update the deployment time is greatly reduced making it less of an issue. It would take some doing to match Capistrano deployment though which is simply awesome and something I will certainly miss.
I actually used a powershell script to automate deployments on Azure, it was the delay from when you do that which is so frustrating. Waiting 5-10 minutes was fairly normal.
If you mix .NET code with significant OSS components (i.e. databases), you will incur a lot of extra setup work and hassle. There is less of a community and therefore you'll have to figure more stuff out on your own. If you're a .NET programmer for life, then this is great experience for you.
One of the traits of a good CTO is to access the community around a mix of technologies (i.e. C#/Sql Server, C#/MongoDb, or Ruby/MongoDb). If there's a good community around your technology stack, that'll streamline development a lot more. Things will just work. Many people will have already solved the problems you're bound to have.
I guess I like to think if my databases grow too large for the free version that's a good problem to have. It's not that bad a "time-bomb" to me. Many sites will never need the paid version - especially the ones small enough to be so cost sensitive.
MySQL is a pleasure to work with and so it MongoDB, more so than SQL Server.
If you prefer SQL Server than more power to you.
Anyway, I hope that articles and interviews like yours do make MS lower the price of SQL standard, but people shouldn't dismiss the MS platform based on cost when in many cases it could be almost free.
I do agree with that the SQL server licensing is a huge PITA and in your case you really didn't need all its ancillary features and instead just needed a simple, fast data store.
The article was a great read.
If you're developing on the MS stack, you simply need VS.NET and ReSharper or you'll be wasting hours of your life every week.
Otherwise, you'll be fighting against the grain. And the support community will be much smaller (i.e. .NET w/ OSS databases).
I agree about using the rest of the MS stack with .net but the DB component specifically is highly modular. Let's say you want to go for PostGre or Couch. Neither of those are part of the 'RoR stack' is it? How is is any less correct to use PostGre + .net rather than PostGre + RoR? Is the PostGre + .net community really that much smaller than the PostGre + RoR? I seriously doubt it.
Run MySQL on Windows and see how slow it is. The driver story is horrific.
My opinion comes from first-hand experience developing for a .NET-based startup. I have faced the issues that the two TekPub guys have. Although, switching away from .NET isn't an option for me. So, sometimes I need to integrate with OSS components. And the .NET examples/docs/community are often sorely lacking, and I have to figure out a lot on my own. I realized that it would be a lot easier to integrate if my employer was a PHP/Ruby/Python/Java shop.
Could you give me an example of the sorts of issues you've had? I've been using .net since 1.0 and we integrate with dozens of thirds party systems every year and database tools for .net are in my experience mature enough to not be an issue.
Small Business Server gives you all the backend stuff for more like $1000 per server. BizSpark/EmpowerISV/WhateverTheyCallItNextYear gives you all the dev tools and OS stuff essentially for free.
I can't imagine that anybody has EVER actually paid $7k per processor for a SQL Server license. We've been running it on our servers for years, properly licensed, and never even come close to that figure.
http://blog.wekeroad.com/2010/05/19/no-sql-in-the-wild#comme...
Microsoft comes out with new ISV programs about as often as new versions of Visual Studio. As long as you join the latest one every few years, you'll always have an up-to-date set of developer tools.
So all that's left is server costs, which as I mentioned above come to a little over $1k/box.
Windows Web Server: $400 SQL Server Express: Free (Databases have a 4 GB maximum size)
I can't image how it would cost them 5 figures to use asp.net mvc. I find the productivity of the MS development stack to be well worth the meager cost.
Take stackoverflow.com - a hugely successful site that I believe runs off 3 servers (originally 1). Jeff and Joel put this together on a pretty tight budget and I bet their site handles a bigger workload than the OP's site.
I think this is a bad article/interview because it gives the impression that you can't do websites in asp.net MVC on the cheap.
We budget around $10k per dev for tools and training per year. We usually come in a fair bit under, but given we spend over $200k/year per dev, this is a small fraction of their cost.
I think your point is that the tools weren't the right fit. Because if they were, it seems like baulking at $10k is a sign that you can't make the right cost-beneift tradeoff -- which is a very bad sign for a company.
Ask Jeff and Joel how much they're going to owe - I think it's almost to 6 figures :).
I wrote a medium-sized online game for a guy who wanted to do it in Mono/MySQL. True, Mono does implement a lot of ASP.NET, and a lot of things work fine, but there are still rough edges that drove us crazy to the point that I finally convinced him to switch over to a Windows server (from Linux/Apache). Some examples:
* Deployment: Mono was very touchy about deploying changes. If you drop in app_offline.htm and then replace every part of the site with new files, it usually worked. When it didn't, it would hang Apache and/or just start returning 500 errors until the web server was restarted. (Whereas with ASP.NET, you can drop in individual files and/or updated DLLs and the site will smoothly recompile as needed.) This was somewhat okay on dev, but got very old very fast in production.
* Strange memory errors: I'm still not sure what was going on, but we had some strange behavior going on where the application would get itself into states that should not have been possible. In one instance I was able to track it down to a particular variable assignment that simply would not happen, despite the code clearly being executed. (E.g., "int x = 42; x++; Trace.Write(x);" would write 42. More complex than that, but that type of thing.) My wild guess is some kind of memory allocation in Mono, but I don't know. Could have been my code somehow, but I have never seen the problem since switching to a Windows server.
* Feature lag: By its very nature, Mono lags behind .NET and it got old being able to use cool 3.5 features (such as LINQ) on other projects and be stuck with 2.0 in Mono.
I think Mono's an impressive project, and my hat's off to the folks to have worked so hard on it, but after some experience with it I decided that if I'm going to write stuff in C# I'm doing it on the Microsoft stack.
I love Mono though and I am actually using that on my other startup (http://adzerk.com) instead of rewriting since I have so much more code there.
I have nothing per se against Microsoft's hosting except for the expense. I believe that it is much more expensive than AppEngine or AWS. Question: the lock-in factor is likely not that big of an issue since apps could be run on Mono and a leased server, right?
Really, software, writing and deploying web apps is a business and there are two large costs involved: labor costs for writing and maintaining software, and hardware+admin costs for deployment. The question is what is least expensive and fastest to deployment. For me, that is Ruby on Rails for the front end, and as needed, back end processing in Java, or Clojure, or Common Lisp - whatever makes sense for a project.