The Real Reason Outsourcing Continues To Fail
lessonsoffailure.com
lessonsoffailure.com
I might be able to help (I'll probably get downmodded into oblivion when some of my fellow Indians wake up but ... ).
The thing is, even in India the best (or even half way decent) programmers don't work for Infosys/Wipro/TCS and other outsourcing companies, who are probably the folks you are dealing with. The second tier outsourcing companies are worse. The good people go work for Google, or Yahoo or Microsoft, (and increasingly these days on their own startups) same as in the USA. The numbers of the latter are on the increase.
The number of really good programmers is as low here as anywhere else. I know maybe half a dozen after fifteen years of working in Bangalore that I would trust my project to.
Yet another (and the most important) factor is that all these outsourcing companies are heavily bureaucratic to a degree a Westerner can't imagine.
Most of it happens because the economics dictate overstaffing. If your revenue model is N people * m $/ hr, it makes sense to increase N as much as possible on every project. Anyone who survives for long in one of these places is by definition someone who practices CYA as an artform. Anyone outspoken and/or willing to admit mistakes will be ejected from the system very fast.
So yes, when you outsource crappy work to India (and most of the work outsourced here is mountains of legacy crap code no one in the West wants to work on anymore, often outsourced without meeting or talking to any of the engineers staffing the project, often on the basis of what some slick salesman put on his PowerPoint deck, yes then you get bad people.
And that is the truth. Your experience is valid and is the norm. Working on outsourced enterprise projects, without hope of ever working on a decent codebase, is soul killing and degrading. You can't expect decent developers or managers on such projects.
When someone replies "Yes sir, I will certainly be very honest with you, sir," that is a good sign you've got one of the bureaucratic ass kissers. "Sir" is always a danger sign. When you hear something like "WTF made you say that Peter? What are you really worried about?" you have a chance.
Yes there are sharp developers and managers here, but it is hardly likely they'll be working in bottom feeding enterprise outsourcing companies. (The director level people at Infosys etc are often very sharp people, which is how they conned you into outsourcing your project to them but that is another story for another day ;-) )
Please don't outsource to India (or anywhere else) before speaking to and attaining a comfort level with engineers and other team members who will be working on your codebase. By the time you encounter these "yes sir, I'll be honest sir" types, you are already lost.
My 2 cents. fwiw.
Let me expand one reason. Bureaucracy. The engineers or even managers you talk to may not have the authority/incentive to be honest with you. The directive from above will be to sugar coat anything they utter.
This has happened to me before :-). I was called a "traitor" (to whom?) and "elitist" for saying something similar to this in another forum.
I don't mind. Karma (even on HN) is just a number on a webpage :-P
You rarely would say "He has accumulated lot of good karma by doing good, I better see what he has to say". On HN, its a different story.
Unfortunately, it is not just a number on HN.
A recent article here revealed a hidden algorithm that makes your votes useless if you drop under the global average karma/comment.
Thus it is more likely that populist opinions have comments shown higher while yours is likely rendered less visible by being grayed out.
It's an issue of freedom of speech, and unfortunately, isn't just another "number".
"Freedom of speech" is not a guarantee that you can post some a comment on some web forum and have it be seen, its a guarantee that if you do, and it is, that your government won't punish you for it.
I realize that freedom of speech in the US doesn't apply to private establishments, but I would expect HN to hold a higher standard than say a conservative Christian forum.
And keep in mind that one of PG's goals is to reduce the herd mentality.
Just because I named 2 dubious uses of free speech, "slander" and "assasination", doesn't mean that the laws can't be twisted to clamp down on more legitimate issues: such as revealing corruption in the government which can be construed as libel in the US.
Overall, I am disappointed in the attitude of HN readers like you who take freedom of speech lightly.
My point is that I don't think outsourced projects fail because "those who build them are not our best engineers". There has to be other reasons.
My job requires me to train my replacement on our APIs if I leave. Over there, I guess it's not the same. When a programmer leaves I'm not given any notice. Just a new contact, who I then have to give the documentation, explain the work, and guide with the work. And this has happened to me 3 times in the last year.
Other than that, the guys are fantastic and good at their job.
Yes the cultural differences are less but in most cases the masking of the problem is still practised, albeit a little more efficiently , which means you find out the problems a little later, that's all.
As someone else pointed out - Good developers are few and far between , irrespective of country and/or culture.
Basically, say "If everything is going great, give me an email when you sign off."
They might be more comfortable that way.
On the one hand, we've had outsourcing projects suffer from cultural issues. However, they're not all related to the high PDI/low PDI distinction -- some of them you could shoehorn into that, to be true, but sometimes when any engineer swears that he absolutely will have a project ready by the end of the day he is just estimating poorly rather than being overly polite.
The number one problem we have had in outsourcing to a particular country is pervasive competence issues, followed by communication issues (egads, no, Babelfish is not an acceptable way to translate engineering documents), followed by a poor mesh in management styles which is exacerbated by both of the other two. We outsource to another country, where we have not had the pervasive competence issues. Things tend to go much better there.
I'm not going to name names, but suffice it to say they're both in that top 10 list and within a few points of each other.
Software development talent is highly mobile, and thus the best people are expensive everywhere in the world.
If you go to a cheap outsourcing company in India who can't afford talented staff, then why are you surprised when they produce poor quality code ?
Let's look at a market where local outsourcing works well: Web development. A huge amount of web development work in outsourced to small web development shops, both by large companies and small companies, and by and large the companies that outsource are happy. Because companies aren't outsourcing to cut costs, they're outsourcing because they don't have the in-house talent and therefore are willing to pay for expensive outsourcing options.
If they decided to go for a cheap local web development outsourcing option (plenty on gumtree/craigslist), they'd get worse quality and be unhappy about it.
Countries like India and China may well have separate issues due to cultural or language barriers, but they'd matter a lot less if the people being employed were talented in the first place.
I find this idea of PDI particularly distasteful because (a) there is no hard evidence so far, and (b) it makes "us" look better. History shows that people believe a lot of dumb things when those things make them look morally superior.
Some countries value individual drive and an equality along all of the power spectrum where those in managerial positions are more like guides than commanders. Other countries have different cultures and values where those in obviously higher positions are deferred to and deserve to be treated with all respect their title confers on to them.
Neither is better than the other, just different. And in order to work better with these other cultures, each side must understand the values held by the other side and reach some sort of middle ground where we can communicate with each other effectively.
I do like your 'hired locally' hypothesis. It makes sense.
I did a lightning talk at the 2009 Great Lakes Software Excellence Conference using that theme. I posted the text and slides of that talk on my blog:
http://jerryjousting.blogspot.com/2009/11/joe-plumber-and-co...
[1] Distributed companies do work, but it is a lot harder to make them work well.
By this logic people who get paid for working on Open Source projects like the Linux Kernel or Hadoop must be bad developers? They are distributed all over the world.
"To see just how much of a factor this is, look at how mishap rates plummeted after the introduction of CRM training at US carriers in the late 1970s/early 1980s. Prior to widespread CRM training, the culture at major US carriers was largely as Gladwell describes current foreign carrier culture: the captain's word was final, and co-pilots didn't speak up or contradict the captain."
Dismissing PDI out of hand as "already handled" is perhaps being disingenuous about the role it may play.
If Gladwell has looked at a few more countries (and perhaps even ploted PDI against crashes) then he's done a much better job than Greenspun's comparison of "Foriegn" and "US" pilots.
If not, then the ball would be in Gladwell's court (so to speak).
It's obvious that Greenspun knows US air safety better than Gladwell though.
Outsourced projects fail for exactly the same reasons that other projects fail. The difference, if there is one, is that outsourced projects are more often set up to copy the exact features that are practically guaranteed to lead to disaster.
Because of their assumption of interchangeability, they end up with a lose-lose model of costs. For illustration, assume a (local) domain expert takes 1 hour to do a job and costs $100/hour. Assume the outsourced inexperienced engineer only costs $33/hour, but takes 3 hours to do the same job because first they have to learn the problem domain.
When you outsource this job, it takes three hours and costs $99. Doing the math, this means the company "saved" $201 because the assumption of interchangeability implies (incorrectly) that the domain expert would also have taken 3 hours.
When you "insource" the job, it takes one our and costs $100. By the math, the accountants figure they left $67 on the table because the assumption of interchangeability implies (again, incorrectly) that the outsourced engineer would also have taken 1 hour.
The only way to win this game is to not play it.
In your example - it was a single task. Working with the assumption that domain expertise can be acquired on the job, the costs of doing subsequent jobs will be lower for the outsourced engineer.
The more times the same task needs to be performed by the engineer, the cheaper outsourcing becomes. Your example just accounted for the startup cost, which can be assumed to be a one-off*
*Is it that way in practice? at least in some projects, it is.
The outsourcing engineers I have worked with have been smart, but inexperienced. [edit] The ones I didn't work with were smart, experienced, and substantially more expensive.
My gross extrapolation from my limited experience with outsourcing is that the engineers gain experience, but that experience makes them more valuable. Since they are smart, they use the outsourcing shop experience as a springboard into a better job, probably not in the outsourcing business or in a "higher up the food chain" (and thus more expensive) outsourcing shop.
Outsourcing is like having a perpetual intern program - all the training costs but never getting the benefit of that training with a long term employee.
This article describes failure of communication which is not unique for outsourcing and can plagues any software project if contious attempts to avoid it are not made.
From his list...
Buyer’s unclear expectations up front - This happens on every single project due to a lack of crystal balls.
Poor governance - I was talking to a guy from a Big 3 US car maker today who was griping about this very topic related to their US ops.
Poor communication - This is in every project failure report I read, US/UK or anywhere in the world.
Poor cultural fit - Have seen this on contracts between large and small companies, although it's usually not that much of a big deal.
Interests become misaligned over time - Apple/Google is a good example of this. Google/Mozilla will be soon. Plenty of All-American examples here.
Not mutually beneficial - Don't think this is just an outsourcing complaint either.
However, I'm going to add some anecdotal evidence about the differences in cultures: when some coworkers interacted with Mexico, they would often say "ahorita mismo" (right now) when they meant "sometime in the future"... if you didn't know what they'd do, you'd have been very disappointed (stuff they said they'd do "right now" got delayed very often).
One problem with outsourcing maintenance to folks from societies that, ah, respect authority to a great deal is that doing maintenance on any large code base will mean deciding that whoever did this the first time around Just Wasn't Thinking, and that you have to rip out and replace large hunks of barely-working, or slowly-working, or maybe non-working code. You just can't have any respect for the egos of those fools who came before you and wrote this drek.
That said, the article's thesis presentation of the state-of-the-industry with respect to outsourcing is very, very well and accurately described. I have seen precisely that sequence of events and communications.
Believe me, this is totally false.
In India (at least around Delhi) you just call any medium size shop, place your order, your order will reach to your door within next couple of hour. Pay the person on your door by cash or credit card (through wireless credit card payment machine). I buy a HP laptop with the same way around 15 days back, where dell was asking for four days for delivery.