Why I don't believe in software architects
mikhanov.com
mikhanov.com
I still code a lot, but I also have responsibility over the whole software ecosystem and hence some power to decide how things go. I mentor developers, perform code reviews, and work on process improvement too. It's a great role.
Thank you for your honesty :)
The bottom line of my post was "beware of the advertise SA positions, they tend not to offer what is usually expected from an advanced position like that".
I also think that a key to being a good SA is to keep writing code, no matter what happens. Looks like this is exactly the case for you.
In my experience, large companies who use Rational will have their architects buried in Rose building UML and writing 150+ page docs all day. Smaller and/or agile/XP companies will be looking for a super experienced coder, who can mentor, and look at the big picture, see the business drivers, etc...
Ideally this sort of thing can be figured out early on in the interview process. "How do your other architects spend their time?" etc...
It worked fantastically well, e.g. I had to stop working on the project for a few weeks to attend to the prototype (that I'd hacked up in in a 6 week burnout), and when I came back the newbie had neatly iterated the design and was able to slip right back into coding with no fuss. By the end of the project my estimates for how long to finish a unit of work were accurate to 30 minutes (by then I'd been programming for two decades, so it wasn't all the process!).
On the other hand, I wonder if Objectory/OOSE/Rational Process solves the wrong problem. If you're going to use imperative OO to build a big complex system, yea ... but maybe you should try e.g. functional programming. I.e. the other relevant development that came out of Ericsson was Erlang and was for the same problem space (e.g. telephone switches).
Still, I feel that if you use parts of it correctly for the right sorts of problems (especially ones that are pretty well understood, e.g. I'd been figuring out for 5 years how to do my project correctly before I had a chance to do it as detailed above) it can be very useful.
The biggest problem in our industry is that there isn't enough mentors to tell the young people that they were wrong all this time.
All the twenty-year-old whizzes I know are experts at taking something like Ruby on Rails, trading in the quality of the underlying framework to put together a hack-job very quickly, and then promptly getting themselves stuck in a morass that they can't get out of because they don't even understand why they are in a morass. Hand these people something with a lot of technical assets they can borrow against and yeah, they can toss something together fast, but that's hardly "better" then what I can do now, which is actually produce technical value.
(Mind you, I'm not saying that's necessarily a bad thing. There's a time and a place for that. What is a bad thing is that the developer won't understand the tradeoffs they just made, and probably didn't make the tradeoff after a sober and careful balancing of the costs and benefits, but did it because it's all they know.)
Anyhow, my point is that thanks to that mythology, a lot of younger people don't think they can be mentored by the old farts they think know nothing in this rapidly changing technical environments. The old farts (at least the ones who have ten years of experience and not one year of experience ten times) have learned that frankly, the tech world isn't changing that fast (different languages, different APIs, pretty much the same messes). There's plenty of mentoring to be done, but you gotta accept that the mentors are going to tell you some things that may shattered your treasured conceptions of the programming world....
Many of us "20-year-old whizzes" don't do what you describe at all. In fact, as a Systems programmer, it's not incredibly likely that you know 'better than me' what the implications of what you're doing are at an Operating System level. I know 20-year-olds who have contributed source to Linux, Chrome, Firefox, and a number of other Open Source projects. Folks who have worked in industry for 2-4 summers in a row already. 20-year-olds who have gone to Silicon Valley and done the startup thing. (I can go on with this BS forever)
But all that 20-year-old ego stuff aside, it's also important to remember that this is a field that is routinely dominated by disruptive thoughts, ideas, and personalities. All of your experience becomes a hindrance when you start to follow your old steps to the "right solution" every time. For that reason, it's also important that you look to the "20-something whizzes" for insights and ideas that may surprise you and that you actually consider what they say.
While it is true that there are many 20 year olds with very impressive tech resumes, many more impressive than mine (I went in the Army and did some other things before entering the tech industry, so I came in older than average). But they are the exception, not the rule. When I interview young programmers, most come as the post described with a CS degree and I hope to work 9-5 having done little-to-no programming that was required. At least from what I see in interviews, those are more common than the impressive resumes.
Next is, All of your experience becomes a hindrance when you start to follow your old steps to the "right solution" every time. This can be true in some cases, but again I think those are the exception, not the rule. Remember that most mid-career people, especially the successful ones and especially in technology, can be quite adaptable.
Also, remember that most disruptive thoughts are older ideas that are just now coming into their own. Lisp has been around for a very long time, but it is starting to take off now (at least more so than it was in the past). Thin clients are a very cyclic idea that once were the norm in an extreme with dumb terminals to mainframes, then gave way to desktops, then were talked about without much success, and now are seeing some success in netbooks/smartphones/etc.
Most of the time the more experienced and seasoned person can adapt and then leverage their experience with similar things even in a new, disruptive idea.
The people who do 9-5 coding are clearly not often the same ones as those who have contributed to the Linux kernel.
What I have found is that, of the people I work with, there are some who are great at what they do. As they gain experience the style of greatness often changes, but the magnitude does not.
(I have even worked with people who become more risk seeking as they progress - they know it can't be done but they don't care as they are confident they can hack around it in time. Cooks especially, but some programmers too.)
As a 1-year junior with crushed dreams by a proto-startup project and now working in a technology conservative environment I can say there is so much to learn in the first years (from bad or good examples, with or without mentorship) that everyone should review his attitude. Of course always keeping the ambition and with the goal of accomplishing the most.
To have success while young and without experience you must be allowed to fail, and to fail a lot of times. Out of frustation of course, I'm starting to believe it's really hard to have success in the early times. Of course close mentoring as in Y Combinator should help a lot!
At the same time, senior people aren't always right, and it's always a good thing to have the fresh perspective that a more junior developer can bring.
I've seen one or two of this kinds at Microsoft as interns. Boy they're really geeky. They have the technical chop. They will argue you to death how pointer works to the lowest-level of your hardware.
With. Exact. Details. And. Unmatched. Precisions.
At the end of the day, some of these people would write some crazy stuffs that a simple solution might work. Why is that? because they were already in "expert" mode in such a young age.
I rarely see "Uncle Bob", "Joel Spolsky", "Paul Graham", "Brian Kernighan", "Michael Feathers", "GoFs", "Martin Fowler" kinds from this group of people. In fact... I know none.
I don't mean to discredit this group. They're smart. But sometime I wonder if they would listen to advises...
Please ask yourself this: even if you do know 20-year-olds who have done impressive things, why would you think that such talented and/or hard-working people will not be able to achieve still more impressive things at 30, when they have roughly an order of magnitude more useful experience? What about at 40, when they have had time to see through several long-term projects?
Or look at it the other way around: do you really think none of the successful older developers today also contributed to volunteer projects during school, worked summer jobs, or tried a start-up straight out of college?
It is true that disruption of routine is sometimes important for the industry to make progress, but it says a lot that you chose the word "dominated". Our industry isn't "dominated" by disruption at all. On the contrary, it is dominated by people who make useful products, using untrendy but tried-and-tested tools and techniques, that get real work done.
Every now and then, something truly original or a genuinely creative application of an old idea comes along, and that can become a great success. But for every one of those, there are a lot of ideas that stink and go nowhere. And guess what? Many of those ideas come from the young people who are convinced they're on a winner, because they are too inexperienced to know better, where more experienced developers might have seen similar ideas fail before and know an idea was doomed before wasting any time on it. Conversely, despite your rather odd assertion that broader experience somehow implies being set in one's ways, IME those who have been around for a while are much better at spotting opportunities to fill useful niches or change the rules in a significant way, and are much better at making practical, pragmatic decisions to take advantage of those opportunities.
YMMV, but I'd bet a substantial chunk of money that in ten years' time, you'll be on my side of the argument. :-)
I was a fairly quick riser in my career. I started programming at 12, had a commercial game (value-ware CD) published at 17. Wrote a book at 21. I was a lead architect at 24, with a development team of around 30 folks at a fairly successful company. I was quite sure that I knew it all at that point.
I didn't. I just turned 30 and only 6 years later my level of knowledge and experience is far superior to what it was then. Not just in terms of pure programming knowledge, but also in terms of decision making. I've become far more practical and much more adept at translating requirements into something people can actually use.
30 year old me and 24 year old me wouldn't see eye-to-eye on a lot of things. After leaving that job I've become an entrepreneur. The sum of that experience makes me far more productive and the quality of what I make is simply better.
24 year old me could a learn a thing or two from 30 year old me. Just as I hope 30 year old me will need a good talking to by 40 year old me:)
While I find it regrettable that the implication of this remark is that I'm some sort of idiot who doesn't have the faintest idea what he's talking about (making him think he does), I understand how you could see this as a presentation of it. I'm not basing my claims off the fact that some of the guys I've met "know everything," I'm basing my claims off the fact that many of the most experienced developers I know don't know shit in comparison.
Granted, it's not a fair comparison of 20-something to 40-somethings at all; I've met far more mediocre 20-something programmers than I've met 40-something programmers, and I'm at a place where fantastic 20-something programmers tend to aggregate.
That said, it's not fair at all for the claim to be made that 20-year-olds are dumbass Rails coders [1] who need to just shut up and listen, because many of us are comfortable operating in a large variety of languages and large projects and are EAGER to listen to worthwhile advice (I probably average a half a post a day? Maybe less? I assure you that I visit HN far more often than that).
All that said, many of the biggest home runs in the short history of our field have come from 20 year old kids who did their own thing.
[1] Which is not, in any way, to imply that Rails coders are dumbasses. It was just the implication of ggp that the 20 year old rails coders are.
However, as impressed as you may be by people with three years of experience, it's still not ten years of experience. (Or twenty.)
May I also point out that you casually assume the other side of the stereotype: 'All of your experience becomes a hindrance when you start to follow your old steps to the "right solution" every time.' That's total bullshit. If you are getting "real" experience, the "good" experience (which I scarequote mostly because I don't want to go deep into trying to define them but it really ought to be obvious if you think about it), then experience itself tells you when your old experience needs to be augmented.
And what disruption are you talking about? I bet in your head you're thinking app disruption; "old fogies" can't come up with the idea for Facebook or something. In programming terms, the disruption is way slower than it appears. When I was 20, I got into Python. Python is still working on penetrating the mainstream. Now I'm getting into Haskell, and if it ever goes mainstream (probably not, but if...) it'll be another 5 years minimum. Programming "disruption" really doesn't move that fast.
(For all the sound and fury of Clojure and Scala and Python and Perl and Ruby and so on and so on, huge amounts of the cognitive traffic are just "An ORM for X!" "A jpeg library for X!" "A JSON library for X!" Patterned, shall we say. True disruption happens way less often.)
Besides... I'd point out that none of what you cite actually disproves my point. Everything you cite involves situations where they were probably either vetted by other people higher up, or still trading on quality to produce results quickly that are still a net technical value loss, as open source does not intrinsically prevent that. Open source even has some well-known common failure cases on that front, like "preference explosion", using adding preferences to prevent having to make design decisions or telling someone to shove off.
You complain as if I'm personally attacking you. I'm not. I'm attacking the stereotype, which definitely exists. And I personally have gotten to the point that someone introduces somebody as a "whiz", I mentally cringe. (The same cringe I do when somebody says "Wow, X fixes like ten bugs a day every day!" BIG RED FLASHING LIGHTS should be going off, not the good kind.) As much as you may not like it... this metric hasn't failed me yet.
There seems to be a large demand for software, and the system is good at producing a lot of software.
as if their code is flawless...
As a federal contractor for an agency that prefers to use COTS/GOTS software, there is a surprisingly great need for having a developer that can look at a given piece of software, intuit how it works, probe its sales and development teams to say "Well, this claims to do Kerberos authentication, but it really does GSSAPI, and hence, won't work with your existing OpenSSO implementation."
I also code a lot, mentor other developers, etc., and yes, it IS a great role.
Sketch out the big picture, take care of the trickiest details, wrangle your underlings into doing the rest.
After some years of this mindset I finally realized that the proportion of excellent people to mediocre ones is roughly the same in every profession. I no longer draw the lines between "hackers" and "suits" (a kindergarden attitude, really), but between excellent people and mediocre people. It's liberating. You can go into any profession with this mindset and it will be your own personal playground.
They claim that only about 1 out of 5 programmers "get" abstraction. To the extent this is true (and it agrees with my experience in the field starting in 1977 including a dozen years in and around MIT) you can't have "democratic" design processes, for the "software architects" who do get abstraction will be voted down and you'll end up with the usual mess of spaghetti (or whatever they're calling it nowadays).
Hmmm ... another problem I ran into was understanding scaling, O(whatever) issues. At least one project I declined to join died horribly because the people in it couldn't grok that their design wouldn't scale to the level needed for basic customers, let alone big ones (in this case the lead had only done CD-ROM based applications, so his library code for the DB didn't even work when they tested 2 clients, but the bottom line was equivalent to trying to use e.g. the biggest baddest version of Oracle to implement a Google scale internet search engine).
Of course architects need to be technically strong, because they need to be working with the team leads responsible for each part of the software project. That means they need to be able to communicate effectively with those leads and understand the technical implications of what is being said.
But it's a two-way street, and both the team leads and the architects have their own role to play. The team lead needs to be more aware of the finer details going on in their area of the project. The architect doesn't need to know those things, but does need to be aware of how everything fits together. That includes knowing enough about what each individual team is doing to identify areas where something new would be useful, or where multiple teams are doing the same thing and there is redundancy that could be eliminated.
Ultimately, it's all about collaboration. A software architect who only draws diagrams and produces hundred-page design documents probably isn't very useful. A software architect who talks to business leaders to clarify the requirements, liaises between the various development teams to make sure that they collectively meet those requirements, keeps an eye on testing and on bug reports coming in from the field, and so on, is pretty much essential to building large software systems successfully IME.
(Edit: I realise that my final description there was rather broad, and has some overlap with management responsibilities. Realistically, I think a software architect is going to have an element of that as part of their role, just as team leads do within their particular areas. But both are primarily technical roles.)
I'm currently in an embedded systems company where the chief architect came from a server-design group. So now everything we do looks like a server. It's awful on multiple levels.
I'm not justifying the term software architect (I don't like it for other reasons), I'm simply suggesting that if your infrastructure consists of a single, monolithic application, you need to consider solving some of those problems outside of your code.
What are your reasons, BTW?
Architecture school is very hard, mentally and physically. The perk is the weird mystique that society gives us.
I think that if you are competent enough to build any part of your project, but can still understand the big picture and have some appreciation for elegance and beauty in a design, you've got the basic idea.
I think architects are also better trained at accurately sizing up time requirements and logistics, for an entire project, and at being aware of time and money and arbitrary client constraints.
And I think architects are trained a lot more in presenting their ideas to hostile audiences, which gives a lot of insight into taking and giving criticism, and finding the flaws in your projects and designs before they ever reach a client.
Understanding the big picture is an obvious requirement (and not necessarily easy ... perhaps more important is knowing when you don't have it, see some of the comments here about architects who switched problem domains), but too many people don't grok the correlation between elegance and beauty in design and how they make projects easier and sometimes just simply possible.
One note of importance that's related to how we aren't trained in those critical not so technical skills you cite is the what we're building is fantastically more complicated than anything concrete (so to speak :-) that others build (note that the most complicated microprocessors have a lot of software (firmware) in them). One key sign I've found when I've gotten a design right is that it solves problems that I didn't know I had. I think it takes a lot of experience to get to that point (by then I'd been programming nearly two decades, and studying on my own software engineering and design from the beginning).
Time requirements and logistics in our field can be very difficult, since you often don't know the scope of the project until you're part way into it. The waterfall process works for you (it had better!) but it doesn't tend to work well for us ... and there comes in the least technical other skills you cite ("arbitrary client constraints", oh, yes!).
One final note: we can mostly get away with so many in our field being idiots because what we do is mostly less consequential, but if you noticed the recent stories about how medical radiation devices are still killing people you'll see we really aren't as far as we should be.
Thanks a lot for your reply.
If you answer no to the last one you should read the Chaos Report.
(Which is a big part of the reason I choose to work as a coder ;)
I am saying that a top notch architecture school is much more demanding than an equivalent engineering school, from my experience. I say that having lived with ME and CS roommates, at U of Michigan, while I was in school.
Talking about aesthetics, Information Architecture, from the data model to business objects model (or what the users call the fields in an user interface), even the the names you give to the applications are a big aesthetic exercise.
An aesthetic exercise where you your audience is the business side, and you need to appeal to their comprehension.
If you're doing anything Prescriptive, you're really doing design, for better or worse. Software Architecture is properly Descriptive -- it's an explanation of how a system works, and how to make your work fit its structure. It's not something you should ever be doing up front.
The GoF had a good idea to describe software in Alexanderian patterns, but they didn't really understand why and for whom his pattern system was designed, and then they fucked up the implementation in the extreme.
Not the sorts who wave their hands and draw nonsense diagrams on a whiteboard and chuck something that cannot work, ever over the fence to the implementation team and blame their skills when the project fails.
On the other hand, a larger organization with lots of different teams probably has a valid need for someone to bring in cohesion.