Symptoms of Dysfunction in Software Teams
blog.hedges.net
blog.hedges.net
[…]
Yeah, that became quite the thing. Lots of new features. Funny thing is, they used to come fast. Not so much lately. Did ask why —were simple features—, but all I got was some bullshit about about "technical debt". Don't give a crap is what they do. No matter, we're entering cash cow mode now. Champaign?
[…]
I'd like to, but John left (something about a "big ball of mud"), and Bob got promoted… That leaves Jane and Sally. Still many glitches to fix, but they will do —not like I have a choice. Besides, they know the billing subsystem inside out, and women are good at cleaning things up, right? Wait, I didn't —are you recording this?
[…]
Phew, that was a close call. Jane warned IT about that OS update, but they kept talking about this "security update". Four days without revenue! Jane and Sally were no good, even Bob couldn't see what was wrong. Had to call John back, you know. Fixed the problem in a pinch. I hear HR made an offer, but I guess there was no changing his mind. Oh well, at least things are back to normal.
[…]
Dunno why, but this systems scares me a little bit.
They did. They wanted it ready in 3 months. I estimated 10.
We delivered something - I'd hesitate to call it a finished product - 15 months later, and it was deservedly panned by users.
I've spent a lot of time since pondering exactly where it all started to go wrong, and I'm entirely convinced the day we demonstrated the prototype was it.
I interviewed there, and turned down an offer. When I was talking with them, I joked that they needed to be careful or they'd wind up shipping their demo code. "Oh, don't worry," they said, "We're going to re-write all of that."
A year later they shipped their demo code. A year after that they'd sunk without a trace.
(I sometimes wonder if they would have had a chance if I'd joined them. I'm guessing it would have turned out pretty much the same).
Maybe if you joined as VP of Engineering or CTO. As an IC, all you're going to do is get dragged down with them, and go home pissed off every night.
I've been there. A company has no existing test infrastructure, it's all done by hand, and the build "system" are the binaries that come off a dev's machine? No problem! I'm you're man, I'll have you up and automated in a jiffy. Except that the reason there's no existing infrastructure now is because of general cluelessness, not for lack of expertise.
You need someone to hook the hoozit that I lightly touched last time I was there to your new whatzit? Yeah, I'll come in on a contract basis to get it sorted. My first clue should be that none of the other devs who know it better than I came to your rescue. No specs for the thing I'm supposed to integrate with, no documentation as to what it is you actually want, and a PM that sits and watches YouTube videos all day? Yeah, I should have guessed.
It's top-down, and if you're not at the top, you're not going to save them from themselves.
That never happens, unless you force the issue by making the first version fundamentally unsalvageable through choice of language or something like that.
If you don't know for sure that you can't (for a very real reason, like the fact that AS3 won't run on iOS) ship a line of code, it's on you to assume that everything you write is going to get to production. That's the reality, even if it's uncomfortable.
...I say as the lead on a game that just got sent in for review today, 8 months early, with 95% of the "prototype" code that we started a few months ago intact. Luckily we never believed that it was actually prototype code, because wasting a month and a half to recreate something that already works is a hard sell to anyone that hasn't struggled with the codebase already.
Pad estimates, and plan for the worst, is all I can suggest. But also realize that sometimes short timelines are a blessing: the people that would be filling up the feature request list stop doing so once you tell them that you're already overcommitted by 80%. That helps the product, usually.
How does X close a TCP connection... it depends on the operating system in question. What cipher does my browser use when talking to X website... it depends on what ciphers are supported/available and how the client/server are configured. Which router do these packets go through... it depends. Is my password secure... it depends on the string you chose, the hash type used to store the password and who the attacker is and what their resources and time frames are.
There is hardly anything absolute in technology/software. And, people who want an absolute answer are only indicating that they do not understand the fundamental complexity issues that we deal with as technologists.
A variation of Conway's law. https://en.wikipedia.org/wiki/Conways_Law
Exciting or sad, or both?
It could happen, but I've never seen it, nor have I even read about it the literature. The sum conclusion of 20 years of software engineering literature is that two things matter: the quality of your programmers and the stability of your requirements. Of those two, the first matters more than the second. Get those two, and it doesn't matter what methodology you use, you'll have functional and elegant software. Miss those two, and it doesn't matter what methodology you use, your software will be crap.
I've always believed that when excellent programmers advocate whatever methodology they use (e.g. Beck et al.) they themselves are missing the fundamental fact that they could produce excellent software using three sea shells.
I'm really glad that those excellent and public programmers don't have a more perverse sense of humor.
... or, maybe they do ...
They might be the best answers you could get in that company, but it would be better still to get:
"Use this library feature"
"Ok; are there any weird exceptions?"
"No, this is it. We rolled all that company's accounts into this system when we merged, all old accounts in when we upgraded, and planned and designed this to be good enough for the high risk accounts too. There's only a handful of other accounts kept separately and there is no way to verify those from here, by design."
One day a coworker and I asked him a question, and demanded that he give us a yes or no answer. He thought for a moment, and then replied: "Yes. However..."
[Edit: Fixed typo.]
http://blog.codinghorror.com/the-big-ball-of-mud-and-other-a...
I've never seen a competent team spend any amount of time chasing after ticketing system nirvana.
I briefly worked at a place migrating from Trac to Jira. Oh, and they still had some stuff in Bugzilla. This team wasn't more than 2 years old.
A month later a totally broken copy of JIRA appeared whilst I was on holiday with the old process that was broken crudely crammed into it with no reporting, no backup/restore and no amount of crazy spared. Yep clearly the problem was the issue tracker not the process.
Now $32000 a year in protection money from Atlassian hits the upper management team (as well as the process brick wall which bottlenecks everything on the PM) so they decided that the problem is:
"We need another ticketing system. Sales force is looking good if we can just customise it for our process."
So I quit with much "fuuuuuuuu..."
(Actually, the inverse error is often made: trying to get it right the first time around, which is impossible. A prototype is even worse, but least you can test your major ideas, so you can see which are good, and which are hopeless.)
The problems start arising when the issues of scalability and maintenance were not considered part of the prototype development. And this problem does not only affect "carcinogenic prototypes" [sic] ... such problems manifest, and do so regularly, with software projects that had multi-year deadlines and good intentions all-round vis-à-vis scalability and easy maintenance.
I strongly doubt they have much unique to software development systems, to be honest.
Dependencies - Check
Jane Doe - Check
Town Hero - Hey look! It's me!
Carcinogenic Prototype - Yeah sure, that's the one Jane Doe is maintaining!
The shitty part is that they pay me above the market.
Source: I was That Guy for about 8 years at three different gigs. Now I specifically negotiate no afterhours support in my job interviews, and turn off my phone when on holiday.
You can gather this information in a number of ways in an interview. Simply ask one of the interviewers where their group exists in the company hierarchy. If you don't want to be explicit, ask to look at the area where the developers sit and work. If the area is well done and people seem happy, you're probably in good shape. If the people are essentially stuffed into a closet or makeshift office/hallway then you might want to run.
Generally, try to work for a company that makes money on software or technology. A company that makes money on other products or services is likely to treat software developers as a low-level function of the organization.
I'd say this is usually good advice, but with a caveat: larger companies with deep chains of command -- even software companies -- typically have "management culture". These types of companies put a lot of pressure on their employees to move into management, even if they're not good at it or they don't want to. These companies reward playing social power games above any other kind of achievement. There will be lots of meetings and not a lot of work being done. There will be no reward for technological innovation or productivity.
So, I'd say avoid companies with deep management hierarchies unless 1) you want to go into management or 2) are stoked by the thought of doing the bare minimum.
To me, it felt like a psychological prison. I managed to break out.
The only advice I can give is this: if you're unhappy where you are, start planning your escape. Start networking and be patient; in my experience, this is the way to land a good job. On the other hand, if you're content at your current company then who cares what other people call dysfunctional?
That can induce bad habits.
People look askance at resumes from successful Microsoft employees because to succeed in the last N years they had to do things that add no value to anything other than surviving or thriving at that dysfunctional company. Why hire such an person if there's a good chance he'll do some damage to your company while you attempt to deprogram him, during which he'll be much less productive that someone from a less damaged company.
(OK, finding the latter might actually be difficult....)
> Why hire such an person if there's a good chance he'll do some damage to your company while you attempt to deprogram him, during which he'll be much less productive that someone from a less damaged company.
Hire such a person if you happen to be hiring for a dysfunctional company. They've got the experience, and they'll appear productive immediately.
So then the "black mark" on the resume only becomes a hindrance if after, say, 10 years a person finally decides to break out. It's still possible, but it will take a lot more work.
There's a lot of technical debt, but I have the time to clean it up. Most of the previous guys in my job just coasted, so I'm already kicking ass compared to them.
Edit: "Technical debt has to be the acceptable cost of rapid and affordable innovation."
We can't talk about (technical) debt without recognising the commensurate leverage it provides to business. If a debt does not continue leveraging the business, then and only then, can it be considered carcinogenic. And the only reliable solution then is to simply switch the solution off or replace it.
The easiest way to avoid these problems is to commit to clean code, and having the stones to stick to that plan. That means someone in power needs to agree with this way of doing things.
In my experience, the tone of a company is set from the top and works down. If the boss is non-technical and is not open to delegating, then there is nothing that can save that company other than new leadership.
What happens is the competent people bail, leaving behind the less talented and/or less motivated which creates a positive feedback cycle.
Well described here:
http://brucefwebster.com/2008/04/11/the-wetware-crisis-the-d...
That's painting with a broad brush, as software is used in a huge variety of businesses. But I'd plan on any organization without mega-income-$$$ (and even that is no guarantee) being pushed to their level of competence and beyond, having turnover to deal with, having features that marketing 'needs quickly to close a big deal', and maintaining zombie ware [it wont' die]. Welcome to my 15 years of experience...
FTFY. Seriously - #2 & #3, not enough money for a 'co pilot'. And for 2, at least there is some ostensible owner of the module, rather than a maintenance by committee which can be even shittier.
#1, maybe not cost effective to move, or would be opportunity cost. #4, I'm sympathetic to not shipping the prototype, but ever hear of first mover / time to market? obsolescence is one of the biggest risks in development[1]. Plan on shipping it and rewriting it in bite size chunks ... forever!
Anyway, it was a nice read. 'mortgage driven development' hah.
[1] In the Mythical Man Month, these are both mentioned, even though they contradict each other, and many more developers seem to only quote the 'don't ship the prototype'.