How Microsoft dragged its development practices into the 21st century
arstechnica.com
arstechnica.com
Pretty good article, only some minor inaccuracies.
Things I noticed:
1) Private offices are nowhere near gone. Some teams have "gone team room" (TFS), some have not (Roslyn). I've worked in both and personally don't care either way, but there are a number of people who heavily favor one or the other.
2) Visual Studio is technically a team, yes, but there are thousands of devs working on it. I think of my team as Managed Languages (Roslyn), not VS.
3) Development strategy is a continuous process and one we're still currently engaged in. We haven't just decided that this is the new process and we're not changing it again.
Oh, and some personal feelings: tooling isn't really mentioned in this article but it may be the most important. TFS and other teams have developed some quite good tools that help with the workflow considerably. If every dev is wasting time messing with bad tools, that's a huge amount of dev time wasted as an organization.
Poor tools decisions or tool apathy (for example, not enough funding to the tools teams) can shackle entire divisions of the company.
They are really great, and they really do make a difference though.
You know, I should have written about that a bit more. It makes sense in retrospect, I didn't even think about it at the time, even after leading off with the problems of the VS2010 cycle.
But, yes. If one looks at how VS and TFS have evolved, it's been very clear that there must have been some internal customer who really wanted to use them for scrum (no external customer would ever be that influential). The last few releases, not to mention the VS2012/2013 quarterly(ish) updates have all had an emphasis on improving this aspect of the products.
Accordingly, they seem to have become pretty decent, and this has no doubt helped teams change how they work.
My view is that you can't develop effective general purpose application software without an internal customer, because until you have that internal customer, you can't meaningfully dogfood it, and you can't get the feedback loop tight enough.
Take, for example, WPF. WPF addressed a real need, I think; developers wanted a decent GUI stack (which Win32/WinForms/MFC/whatever just ain't), and long term, Microsoft wanted something amenable to 3D acceleration and DPI increases. Great.
Except that for the longest time, WPF was deeply problematic for any major application. Problems around text handling and performance were the biggies. Microsoft had no real internal customer for WPF. Some of the Blend apps used it, but they (now discontinued) were never a critical part of Microsoft's day-to-day operation.
When did WPF get good? When an important application (Visual Studio 2010) started using it, and made clear that hey, something needs to be done about that text rendering and performance.
I think there are quite a few little corners of Microsoft's software in a similar boat. The new-to-Vista sound card/audio APIs were plainly not fit for purpose in their original form, but nobody in Microsoft noticed, because nobody in Microsoft was actually using them. Late in the development cycle the APIs were updated (adding notifications when a buffer was nearly empty) to make them usable, but it created a troublesome driver situation for a year or more post-release.
With the internal customer, the feedback becomes much more immediate. Microsoft's customers don't get to see every 3 week cycle. Internal customers do.
When the developers are using the very product they're developing, this becomes better still; there's an implicit understanding of what the thing should do, and how it needs to be better. This becomes institutional knowledge, and it likely never even manifests as an explicit request. It doesn't have to. The implicit understanding of how customers are using the software and not just what they want but _why_ they want it is sufficient to ensure proper and effective implementation of their requests.
> Even as Microsoft stuck to its waterfalls internally, Visual Studio was used by third-party developers who themselves wanted to use agile methods, creating pressure on the Developer Division to build better support for these approaches.
These features are definitely being driven by external customers. (Disclaimer: I work on the TFS team, in particular on the Version Control team, so I can speak to the changes in version control throughout the years.)
TFS was initially conceived as a mix of the tools for software development (version control, issue tracking, continuous integration) that were heavily influenced by the tools already in use at Microsoft to build (say) Windows and Office. That's why the version control component in the first version of TFS was a checkout/edit/checkin model. Clunky, yes, but scalable to the sizes of the codebases that we wanted to support internally. That's also why the work item tracking is crazily flexible in ways that are barely comprehensible to mortals, it was meant to adapt to the different development practices that go on around these parts. So TFS 2005 looked a lot like the development tools that existed within Microsoft at the time, and it was really aimed at "influential" internal customers.
And that's great, but most of our customers outside of these walls don't have crazy multi-gigabyte source trees and the same checkout/edit/checkin model that's so useful for a code base the size of Visual Studio is an unnecessary annoyance if you've got a much smaller or more modular code base. And if you have a dodgy network connection or you travel alot, it's downright problematic. So bringing an edit/merge/commit model ("local workspaces") and a DVCS (Git) to the party were truly about growing and adapting TFS for our customers, who tend to be a little lighter weight and more agile than we are.
There's also in my opinion a huge problem in that what sort of technology you can use is severely restricted by what Microsoft makes. I'd get push back for asking for a license for something like reflector (yes we had a ilasm dissambler but it was horrible) let alone something like recommending using selenium instead of some horrible, under maintained internal solution.
Then there are just the utterly ridiculous political restrictions on the development pipeline. "Oh you have to use Razzle or CoreXT instead of TFS because even though you're a html5 website with SOA components your team is technically part of windows and they use Razzle."
Here we are 10 years later ...
IIRC, CoreXT didn't handle some really oddball scenarios we had that involved self-hosted compilers and runtimes. e.g., rerunning the build with the compiler you just built or just the runtime you just built, or...
We were also trying to shift as much of the managed parts of the stack as possible to using MSBuild natively, which required a custom local runtime build to support asmmeta (which rudi and I created to solve the circular build dependencies in managed libraries in VS).
We were also using subtly different versions of build.exe whose parallel build support and support for scenarios with files over 2^31 bytes differed in non-trivial ways.
In the end, I think I stole everything I liked in CoreXT (which was a lot!) but we couldn't really adopt the workflow without breaking basic requirements that VS had that nobody outside of a compiler toolchain really sees.
(BTW, I apologize if some of this is off. Like you pointed out, this is more than 10 years ago.)
First, say hi when in NYC, dblock at dblock dot org.
I think workflow, spirit and approach are the real deal. Everything else is just software :-)
- Microsoft's private offices were an innovation in the 80s/90s, cramming large numbers of worker bees into a single large office is a retrogressive step, not a 21st century innovation
- The writers focus is on Visual Studio, but a number of other teams had switched to agile much earlier. I worked in Windows Live Mobile in when we switched to "agile" in 2004. This wasn't being dragged into anything, it was a relatively early adoption
- Software projects do often get delayed, but not always. I was one of the dev-leads who worked on Office XP in the 90s and though we had a long ~3year, ship cycle, we did ship on the original planned shipped date (RTM 3/2/1) Steven Sinofsky deserved a lot of credit for that.
http://www.theatlantic.com/business/archive/2011/04/working-...
I’ve also seen some research (which I cannot recall right this second) - which suggests team offices of 4-8 are ideal where you get some of the communication/team productivity without the full disruption of an open office.
Our tool of choice is Sococo, because its awesome and because we develop it (I work at Sococo).
Getting interrupted every 15 minutes while trying to get my head wrapped around some difficult bug fixes makes work virtually impossible.
For every startup I've had, I do all my work from home in my own office. Once a day meetings to go over issues, strategy, etc. makes my days exponentially more productive.
With notebook computers and mobile phones, there is no reason to chain an employee to their desk. A software company could have areas with open tables and school desks designated for a particular working environment. It would require some sound proofed rooms meant for demos and discussions, and at least one large area you could call "The Library" where talking is met with a chorus of angry shushing.
This is just an idea that’s been rattling around my head so I am sure there are points to work out in implementation, but I doubt it would cost significantly more or hurt productivity any worse than open office plans.
What kind of "agile" were you practicing? I don't know about "Windows Live Mobile", but until recently the remnants of Windows Live were pretty much all in a waterfall-esque model: Multi-month milestones with 6-8 weeks each of planning, coding, and stabilization. This was better than 3-year cycles, but I would hardly call it agile. I guess on the full spectrum it's more agile.
The [very] successful process of developing the Linux kernel with nothing but git, mailing lists and a small set of simple rules (no meetings, no Scrum, no BS) proved to be good-enough.
It is not only Linux kernel, there are hundreds of other "decentralized, no-water-fall" projects, notably FreeBSD, OpenBSD, Xorg, LLVM, Golang, CPython, Ruby, you name it.
It is much better to consider the differences between a commercial organization (a corporation) with all that bureaucracy obsessed with keeping its positions and budgets, etc. and a team of enthusiastic professionals with their own "inner" motivations and goals.
Corporations are producing products to make profit, while teams of enthusiasts are producing (evolving) tools and services for themselves.
The difference is the same as between McDonald's and family/home-made-for-themselves food.)
Consider, for example, LLVM/Clang (backed by Apple) as high quality and no-cost alternative to VS. Its used as a primary compiler for OS X, iOS and FreeBSD and optional one for Android NDK.
And, look ma, no bureaucrats, no meetings, no Scrum.
Like you said, a corporation is not the same as a team of enthusiasts, so it seems sensible that what works for the former doesn't work for the later and vice-versa.
OK, let's consider only compilers, if you wish..)
Btw, GNU Emacs, which is another "miracle" of software engineering, is a good-enough IDE part for Clang. Together, in pair, they are, in my opinion, even more flexible and less resourse-wasting than VS. And (surprise!) cross-platform.
The "miracle" of Emacs in that it has a very powerful DSL which uses proper abstractions - buffers, paragraphs, blocks, strings, fonts, characters etc. embedded into a Lisp. Thus it is a thoroughly programmable (not just extensible) editor.
It is quite beneficial to get familiar with ideas and design decisions behind Emacs, especially in the age of Java and copy-paste based coding.
(As a text editor Visual Studio isn't great but it's perfectly adequate and I generally get by by copying and pasting to and from Emacs if I need to do anything complicated.)
I could have a flexible, programmable good-enough DE for C/C++ with make, emacs+cedet+cc-mode on my netbook with a crappy AMD x86-64 CPU and 1.5Gb of available RAM. It works fine for navigating over my own code, when I know what I am doing and why.
I would not explain the wonders of emacs+slime+cmucl or emacs+cider+clojure-mode here. Just one hint: everything works on a [remote] text terminal via ssh,tmux,etc. Google does it better.
Eclipse, while it could start without any progect in a minute or so, is unusable for anything but menu navigation.
Moreover, there are people around who still believe that make and command line tools are still good enough.
My bet is that very complex software, like nginx or postgresql has been written in vi or emacs.
In manys eyes it is quite the opposite: Netbeans and Eclipse used to run circles around VS. I understand a lot of people like VS and there have been improvements lately bur please don't spread FUD on HN. Thanks.
Having been a user of all of them for... as long as they have existed (oh God)... Visual Studio has been my preferred software at the time that each version existed. Even the much-maligned Visual Studio .NET 2002 seemed relatively faster and less error-prone than the then-current release of Eclipse. The decision to use Eclipse was usually a cost or programming language issue. You don't code Java in Visual Studio. There were many times, if I could have, I would have plunked down the cash.
Today, I try to setup my projects to be editor-agnostic, but I still end up using VS Express 2013 for Web when I'm working from a Windows machine. When I'm not, I'm not using Eclipse, I'm using something much lighter, like Vi or Geany.
The world is made a better place when people post from a wealth of real world personal experience and cheapened by those that who do not. Please refrain.
Or XCode, that's Apple's (free) IDE for Mac OS X
I personally love Visual Studio. Using continuous integration was an eye opener to me, I'm still not sold on continuous deployment though (on call isn't a good option IMO for a developer).
Infrequent deployments also greatly increase the risk of problems during deployment, because infrequently exercised pieces of code (i.e. The deployment code/scripts) are more likely to be buggy, and because gobs of changes pile up between deployments.
With that said, continuous deployment probably should not be happening at night or on the weekends unless you have rock-solid automatic rollback and the necessary monitoring to trigger it.
But it only works if you have the time and resources to react to what happens. For example if you find out that differences between dev and prod servers can cause problems, you could add a tool like Chef/Puppet/Ansible to prevent those differences.
But if you don't have the time or resources to invest in that sort of change (as in many government agencies), then continuous deployment will just pack more problems into a shorter period of time.
This would be analogous to lifting weights, but never getting enough sleep or good nutrition. Under those circumstances, a person will just get weaker and more injured over time.
Ditto, though we eventually adopted a modified form of scrum for the project that I'm on. Is the decision out of your control?
Transition from Centralized to Distributed VCS: A Microsoft Case Study on Reasons, Barriers, and Outcomes, via http://research.microsoft.com/en-us/people/nachin/
Have Agile Techniques been the Silver Bullet for Software Development at Microsoft? via http://research.microsoft.com/en-us/people/bmurphy/
[ $0.02, Scrum or no scrum, for my money having a private office is infinitely better than open plan for getting stuff done. Each to their own, getting harder to find jobs with your own space ]
When Microsoft had a dominant market position it could freeze its customer's in-house strategies by tactically leaking details (real or perhaps not) of its future releases. Extended release intervals did not matter because customers waited.
Fast forward to today and it is not dominant in markets where it sees growth potential. Customers will no longer wait. Microsoft is seen seen as a laggard by many adopting new technologies. Extended development time would preclude any chance they have of success so they had to change. The outcome is as yet undetermined.
If I ever have another company, if I can't afford the office space, I don't need the person. Office space is cheap compared to salary and wasted productivity.
Life is better now I work from home. It's better to have the kids screaming in the background than people tapping your shoulder and blasting your entire thought process out of the window.
I was miserable working by myself in a private office.
However, in the case where the open floorplan is simply an office cost savings measure that puts mixed goals in a small collocated area, then yes, headphones..
(Disclaimer; Microsoft employee without a personal office, quite jealous of those with.)
Open offices can only work, IMO, when there's a breakout room, or in general some sort of closed door room anyone can jump into and get things done or have a 4 person meeting. There's just no escaping the need to have privacy for the point of thinking, or even privacy for the point of talking. Maybe there's a perfect job in a perfect world where everything is open, but in my experience, there's a need to get away from the group once in a while, either alone or with a group.
There are these sort of offices available, but the problem is the cost of the context switching, a distraction being an involuntary context switch.
So many people focus on open vs private offices and miss the whole point of which one is better for when.
Realistically, getting the choice seems to be more encouraging.
I'm, unfortunately, at an open office as a cost savings measure type place and without headphones I would get absolutely _nothing_ done.
But you'd probably go insane in my working environment, which is in a house surrounded by fields miles from a city with no one else in it, except me.
Point is, you should tailor the spaces to the individuals working in them. There is no Generic Industry Standard Employee. Which is why trying to enforce Generic Industry Standard Working Environments is maybe not such an effective way to get cool stuff out the door.
As for Agile etc - it's a 20th century methodology, not a 21st century one.
I have no idea what 21st century software development is going to look like. Nor does anyone else, because no one has invented it yet.
I believe the last role took the biscuit, though....
Open plan with hotdesking and lockers... Primary coloured lockers, so it looked like a poor man's GooglePlex. Where the meagre budget had run out halfway through.
You could never be sure where you would be the next morning, or next to whom (inc. constantly teleconferencing middle managers). There were two banks of desks with dual monitors, supposedly reserved for developers. Sometimes, non-technical personnel would grab them, and not even use the monitors... Just their laptop screens (for Outlook, basically).
That was slightly irksome :-)
Good for you. Not everyone can.
I work in an open plan office. I don't wear headphones. Yes, it's annoying. Yes, it's distracting. Yes, it's better than losing my hearing.
http://www.earplugstore.com/ultrafit25.html
When I'm feeling like I really need quiet I'll throw on a pair of ear muffs on top of that. The only caveat is that you need to be comfortable with silence: the ear plugs will really cut down on background noise (e.g. AC and TV).
Thanks for the idea. :)
You can get a good 10db attenuation from just closed back headphones alone.
I've been listening to headphones all day, most days at work for 15 or so years. No hearing problems.
Well, none that can be put down to headphone use, last time I got tested apparently they found conductive hearing loss in my left ear and asked if I'd had a head injury o_O
In my experience the levels you need to drown everything out (to a manageable level and with closed headphones) is way lower for white/brown noise than it is for music.
Also, noise cancelling headphones can allow you to listen to music at quieter than the office's ambient volume.
Google for "more productive at coffee shops".
That would seem to suggest that maybe private offices aren't quite always best
I suspect whether one can work better with some background 'noise' (sounds and sights) is partly a personality characteristic, but that interruptions are universally disastrous when one tries to be productive.
Culturally, I think it's important to notice that VS is really a bunch of smaller teams put together, and not all of them have the same development strategies or systems.
So the biggest motivator here seemed to be a few pioneering teams who thought they had a better way to develop software, and had support from leadership.
So the pioneering teams pushed forward and started to put up measurable results. I think after that culture is bound to follow.
Edit: I left Microsoft to escape the VS ship schedule, I have not heard of significant changes from the co-workers I keep in touch with.
I think it's more likely that the early adopter teams had some measurable result, and then management pushed the change through to other teams. So how did those teams react?
The article just paints a very optimistic picture. Something as big as VS, there's a lot of stuff to actually change when you go about changing things. I feel like there's just more to this story.
People underestimate Microsoft. It has enormous inertia, but historically enough introspection to recognise strategic errors and reorient. The most famous being accepting that the Internet was not going away, circa 1998.
Everything about methodologies should be taken with a grain of salt, what works somewhere, with a specific set of people and for some projects is not guaranteed to work anywhere. And not every aspect linked with "old" methodologies should be viewed as pure evil.
Consider that many of the ideas that ended up in the agile manifesto could already be found in Peopleware from the eighties and that no methodology is the last. There was Waterfall, RUP and other clones, with the Agile wave someone defined XP (now dead) and after that Scrum. Now even Scrum is seen as something with too much overhead and people start turning to custom solution that use kanban/kaizen/etc...
[1] Referring to a project that has some logic worthy of benig explained, interaction with external components, ecc... not some CRUD.
"That's 20th century thinking, Peter! Marshall McLuhan would be disappointed! https://www.youtube.com/watch?v=NNhRCRAL6sY
Spontaneous collaboration can happen just as effectively with a remote staff, but you have to default all your channels of communication to digital ones.
Instead of a culture of shoulder taps, use IM.
Instead of a culture of hallway conversations, get the entire team on a permanent group chat.
Skype is one of the best tools for this, so it's ironic that the company which owns it isn't making the best use of its own tools! ;)
The guys over at Basecamp did a great book recently on how to do this remote collaboration stuff right called "Remote: Office Not Required" http://37signals.com/remote
It's a great read. I read it recently while on the plane to a meeting that totally could have been over the internet. The irony was not lost on me.
It's also worth noting that Agile/Scrum is not universally well liked anyway. In fact it's incredibly divisive. People who love it seem to really love it and people who hate it seem to really hate it.
For example, here's a site parodying the Agile Manifesto, asking the world to adopt something more modern: http://asyncmanifesto.org
And here's a much saucier (and perhaps less serious) take on the same idea: http://programming-motherfucker.com
TL;DR: Agile/Scrum is a management fad. Collaboration can succeed or fail using any methodology. All the blind praise the media spews for Agile/Scrum is arguably harmful to discussing what is and isn't effective management."
Original post: http://arstechnica.com/information-technology/2014/08/how-mi...
So, if you handicap your local staff to use only a fraction of their communications capability then your remote ones are just as good?
Seriously - I've spent a large amount of time dealing with offshort staff and onshore staff, and staff both in the different buildings, the same building but different rooms, and staff in the same room - and having people sitting right next to each other makes a massive difference to the communication bandwidth.
The ability to turn your screen towards a co-worker, say "What do you think of that?" and get a reply three seconds later is _fantastic_ - and simply impossible with remote communications. (Where you can ping them on IM, wait for them to notice the message, start a screen-share, wait for them to accept, and then wave your cursor over the bit of screen you want them to pay attention to. It's effective, but it's nowhere near as fluid as pointing and handwaving.)
As for the workflow you describe being "impossible with remote communications," trust me, it's not! Just paste a screenshot into a group chat.
Often this is faster and more effective than physically showing someone a screen anyway, plus it broadcasts the info to a potentially larger group.
As for the "wait for them to notice" part, that's just bad collaboration and it happens whether people are there or not. If you're expecting your coworker to be interruptible, s/he can be whether it's IM or it's a shoulder tap if that's the expectation.
Meanwhile, in a big crowded open office plan, often times people are just as uninterruptible as remote is so often perceived to be. People put on headphones and strongly resist shoulder taps.
Whether you're all in one office or all collaborating over chat, success or failure is about proper communication, not which medium you're using.
Then, sometimes, you have cases like yesterday where CTO comes in at the end of the day, reads a chat log, and flips out because he thinks we're going to rewrite his garbage co^H^H^H^Hprecious baby. Some nuance can be lost just skimming over chat histories.
[1] https://en.wikipedia.org/wiki/Criticism_of_Microsoft#Vendor_...
I guess downvoters are just upset about MS criticism, rather than consider this being off-topic, since it isn't.
As long as you're guessing, you can say anything you want. It does not have to be true.
Personally, I would not be so arrogant as to claim I can mind-read. YMMV.
I.e. if you disagree with some statement - say so, and explain why. That would be meaningful.
> the problem here is not that Microsoft criticism is unpopular on HN
That depends. According to my observation, a lot of times downvotes happen because of disagreement, even though they supposedly should indicate something else.
Only in markets with poor competition. I.e. in unhealthy markets.
MS is used to such situation, because for a long time they were in a very sick market, where they dominated heavily. Lock-in was used to preserve that domination. Now the situation has changed, and them sticking to lock-in actually harms their position more than helping them to preserve their grip on the market. They slowly realize that, but they could do it faster. Moving slowly has become their habit, exactly because of the above.
Competition does wonders. Remember the times of heavy IE lock-in tactics? They are gone because of the competition.
This is blatantly and obviously false. Mobile has lots of competition, and lots of lock in. Apple's App Store, Google Play Services on Android, DRM on Kindles, DRM everywhere. I can watch Amazon Video on iOS and a Kindle Fire but not on an Android Tablet. I can get a gmail app on iOS and Android but not Kindle Fire. It's lock in everywhere. On Apple's ecosystem, on Google's, on Microsoft's and even on Amazon's.
Not enough really. Current situation of two heavily dominant participants is not called a lot of competition.
Other competitors are too much behind, barely making any traction in the market. And it shows. For example getting native drivers from hardware manufacturers is close to impossible, unless they are requested for existing incumbents. That's a perfect example of implicit lock-in caused by the lack of competition in the mobile space.
Another example are mobile browsers which are dominated by these incumbents. Again, not enough competition there.
Now you're just devolving your argument into a No True Scotsman fallacy. Mobile has a ton of competition, Samsung just lost the top spots in China and India. If anything it's lock in that's preventing it from being more competitive if anything.
And? Samsung's own OS is barely registered on the market (Tizen). Which demonstrates poor competition on the scene dominated by Android and iOS.
> lock in that's preventing it from being more competitive if anything.
As I wrote above, lock-in can turn around and bite the one who is using it. It's a crooked practice.