The IT Productivity Paradox (2015)
cs.stanford.edu
cs.stanford.edu
My ex was good at working airline ticket price periodicity and hotel bookings. We were at an age where all our younger friends were getting married and rather than buying them a wedding present, she would save them $500-1000 on their honeymoon instead. She probably should have been consulting for second-gen travel websites on how to systematize it. But now I think we should have kept the human.
On one hand, it seems ludicrous that (assuming some minimum threshold of personal expectations), "everyone" has to learn "everything" in order to even be competitive. This seems odd for a number of reasons IMO: 1) This is the EXACT OPPOSITE of the specialization efficiency gains in economies of scale, etc. From the story about about automotive production, or making just about anything. 2) This has, for many, only recently been possible. 3) Even if "possible", it is still a WHOLE LOT to learn. Far beyond the (current, believed, practical, or desired) capabilities of many.
I think (1) is a reason why a lot of things can seen subjectively "worse" for those who have been around a while. A lot of things are "better", too, of course.
I conjecture a root cause might doing the cheapest thing for short term profit maximization instead of doing the thing most harmonious with the endeavor in the long run (which could be hard to impossible to stomach in a world of quarterly bonuses).
Personally, so far I have consciously chosen to not participate in being a full stack anything person. I have a niche, it is my sweet spot. I can report that this approach can be difficult. It is mildly successful but not greatly so. And given these trends, I think I'm going to figure something else. I frankly don't wish to know and manage everything about all systems (and I also don't think that's necessarily a great idea in many/most scenarios). That some people do is fine with me.
Another sad thought is that these "do anything" people are completely replaceable in the interchangeability sense, even if it costs a pretty penny. And market forces are pushing everyone (conceivably capable) to become "everything people", hence driving cost down and theoretical corporate (though not necessarily human) resilience high.
What are we to do about it? Just adopt this as beautiful, that one can do so much, and how limitless the mind is?
It's not even a matter of having a paradox of choice. It's that you now often have to spend hours becoming a mini-subject-matter-expert in, e.g., USB-C chargers to make sure you purchase one that won't set the house on fire and is sufficient for your use-case.
Consumer publications & websites (e.g., WireCutter) are in the pocket of BigWidget, so they can't really be relied upon. RTINGS is one notable exception, but I suppose it's just a matter of time with them too.
People with too little leisure time are screwed by this.
"Findings from 3 experimental studies starkly challenge this implicit assumption that having more choices is necessarily more intrinsically motivating than having fewer. These experiments, which were conducted in both field and laboratory settings, show that people are more likely to purchase gourmet jams or chocolates or to undertake optional class essay assignments when offered a limited array of 6 choices rather than a more extensive array of 24 or 30 choices."
More computing power and tools gives us too much choice in how to use the tools: we've got dozens of software packags to choose from for every tasks, dozens of setups, etc. The users (workers) have to constantly choose how to use their tools and which apps to open for the day, etc.
It's the same reason why people love minimalist writing apps. Their computer just presents too many options. Some people have even gone back to typewriters or writing by hand.
Computers aren't really giving us too many benefits. If you closely examine the situations in which computers are purported to improve something, the only thing they ACTUALLY improve in a lot of cases is the advancement of new technology. If you're a programmer or a software company, then new IT is good. But for people just using it as a tool? It sucks.
While computers of course have a ton of shortcomings and difficulties, they allowed easy access to and sharing of information. Collaboration has become much, much easier. To say nothing of plain communication, by text, voice, of video calls. If you think computes suck, try going 50 years back and do a long-distance call (and then pay the bill).
The same thing immensely powered commerce, both online stores, and plain brick-and-mortar stores where you pay by your card. I suspect that productivity gains in the commerce / retail sales area should be very noticeable.
The general meager observed productivity gains may be a result of the way they are observed (much like Amazon is a company with near-zero profits, such a terrible malinvestment!) and how they are distributed. The latter may mean not just siphoning monetary gains, but also changing patterns of control within corporations.
* Easy access to information: perhaps that makes us ignore the most important information right in front of our faces?
* Easy communication: perhaps that has made destroying communities easier (just move away, I can still call you!)
* Online stores:... too much consumerism!
Computers can do great, but I think it required some thinking about what you want to accomplish and how a computer can help. Then tailor your computer usage to that. For writing, you may want something like nano/notepad or the full word processor experience. But don’t use one when what you need is the other.
Choosing may be difficult, but a period of reflection will help tremendously.
I suspect if we were any good at accounting for this stuff, we’d find that 95% of applications of computers have a 0.1x-1.1x multiplier on whatever we’re applying them to, once you factor in the extra costs, but the other 5% is like 5x-500x.
Beneficial overall and yet harmful a majority of the time one encounters it.
Also, ballistics. Modeling/simulation. Data storage and retrieval (think DMV, social security, IRS, banking).
CNC machining and general manufacturing/design.
If you think computers have not dramatically improved nearly every major field of endeavor you interact with, it’s because you were born after they’d already made many major improvements. Or at just ignorant.
For instance, it’s still entirely possible to do manual machining and drafting, but there are very very few situations where it makes sense. Hobbiest and one off jobs, mostly.
Are there areas where they aren’t really helping? Sure.
But they are outnumbered by dozens of ways they are.
People inside the company with domain knowledge and who work with the tool daily cannot contribute anymore.
This is compounded by the process being automated being the 'idealised' one - not taking into account the exceptions and variations.
This leads to the 'computer says no' scenario - where people who used to be empowered to fix something can no longer do so because the software doesn't support the operations.
There is also another problem - often during the implementation of the software solution - the administrator type people tend to pile on lots of requirements for recording stuff that was never previously recorded because the metrics might be useful and the new systems creates the opportunity to capture the data.
This creates extra busy work at best - classic example is timesheets - most data entry into company timesheets ( unless you are the sort of company that charges by the hour ) - is a work of fiction.
Even worse the 'data' gives upper management the feeling that they can make decisions on the data without talking to those middle managers who actually know what's going up. As a result it leads to poor decision making and an increasing breakdown of trust.
It's amazing how much money senior management will spend on an IT system, a sink costs into data entry, in order to avoid a 5 minute conversation with somebody.
Too often, the focus is on what computers do best only.
On the contrary, generally tools such as Workday support a generally ideal workflow, but buyers spend years "integrating" aka devolving that workflow to mimic the manual toil of before the tool, including often unnecessary exceptions and variations.
The end result is often strictly worse, as the flow did not improve and now there are even more human (error prone) steps and more technical parts to break.
I'm suggesting the process overall needs to be able to deal with these exceptions and variations - and it's people who are best at dealing with these fairly ( say in a Workday HR context ).
What you often see is the implementation of an idealised process in the tool, with no way to deal with exceptions - which leads to 'computer says no' scenarios. Everybody knows what the right thing to do is - conceptually it's simple - but nobody seems to know how to make it happen.
At least, that's true of Workday, and most other SaaS I've helped enterprises install.
People don't want that. They need a new tool because the old one generates nothing but complaints, then they want the new tool to do exactly what they used to do. So it too generates nothing but complaints, after the pain of transition.
But what's a SaaS to do, refuse the revenue and share multiple? Right.
The whole challenge with software ( and why it keeps getting written again and again ) is the challenge of balancing flexibility with usability. There is difference between whether the software can technically do something and whether people can achieve that task using the software day to day.
The other problem with systems like Workday ( and this isn't Workdays fault per se ) - is that they are used as a mechanism to 'cut' HR operational costs by simply moving the cost from HR to everyone else via 'self-service'.
Actually most people want to simply talk to somebody - explain what they want and have somebody make it happen - 'self service' only really works for frequently done things where the time saving offsets the time taken to try and work out how to do the operation in the tool for the first time ( or the first time in ages ).
Same thing in finance systems - when large companies move from a purchasing team to a 'self-service' model - most line of business departments simply recreate their own purchasing teams as people who order stuff infrequently have no idea how to fill the forms in.
As I said before - it's about thinking about the people in the system, not the tool. Thinking just about the tool leads to tool implementators blaming lazy users and users blaming tools.
The expertise in the ruleset also falls away over time as people become increasingly reliant on the software system. Eventually the software becomes the definition of the rules, and any changes to the rules have to be changes that someone today can figure out how to implement. If they can't implement it, the rules can't change.
Then you get a big-bang rewrite because everyone is frustrated (or they move to SAP, and are frustrated for other reasons). The rewrite neglects a bunch of edge cases already accounted for in the old system, it takes a decade to finally retire the old system (hopefully), and then in another decade they repeat the process because the expertise is, again, lost.
When you bring in IT, there are inefficiencies. Things that were done without IT have to be done with IT, so there are steps to prepare the input for the machines, and manage the information, and keep backups and all the rest. Some of those activities are new, not replacing another process. Or not replacing it with something less time consuming.
A clerk armed with a room full of filing cabinets won't necessarily be able to handle that many more requests in a day, if that is replaced by a terminal and database. Suppose that during the average request, he has to spend 7 minutes talking with someone on the phone, of which one minute is spent with the filing cabinets. If we replace that minute with 30 seconds of working with the terminal and database, maybe that will go to 6.5 minutes. Or not. If the cabinet or database shuffling is done in parallel with chatting on the phone, it may make hardly any difference at all to the duration of the service episode.
You will not get anywhere near the theoretical gains from technology, with its blinding speed, until all the human steps are replaced. If there is a "for each request x do ..." loop in the operation, and each iterations has steps done by a human, you will not speed it up until you get rid of those steps. But at that point, the efficiency gain will no longer be attributed to a working human as productivity increase.
For instance, there is no question that the web has greatly increased the productivity of corporations in many customer service areas. The farm of servers works unattended, serving up all sorts of information to the users. Thousands of concurrent users. It does something that was impossible prior to the advent of the internet. Because no humans are actually sitting there serving the web requests, there is no human productivity measure.
We can't easily measure productivity gains in companies that didn't otherwise exist (small businesses that can't afford many mathematicians). And for big companies that could afford hundreds of bookkeepers, the gains would drag for years or decades behind since they would transition very slowly.
I'm not an expert though, so maybe I actually just don't understand the issue...
Like an artist who gets a contract to paint a house for a client but when (s)he arrives half the canvas is already wrecked by an offshore team... this is why teams have to put SWE candidates through 9 interviews as a single bad hire risks tainting the canvas for their colleagues
I think current tech interviews (with unpaid take home exercises, live coding, tech quizzes, etc.), are just a really inefficient and ironically lazy way to decide who to hire.
A company could temp-hire 3 different people (anyone who applied and could talk about their past projects), pay them for 1 month and you evaluate them then. At the end of the month, you decide who to keep.
You might fail to pick a great candidate or maybe not but at least you give people a fair chance to get assessed in the job and not on a little dance everyone needs to do just to get into the door, which just wastes everyone's time.
That also allows the candidate to show how they truly are, both skills, learning a new workflow and culturally. Right now, it's just a matter of performance and luck.
/rant
But who would agree to join a company that has a procedure like that?
Most people looking for a job are either already in one, or have several ongoing job applications, or both.
Juniors. I certainly would with my current situation.
I would at least I get some cash during my job hunt, get to make new connections and have a chance to get a new job. Also allows me to evaluate the company and not accept the full time offer after the probation time has expired. Win win.
And yes, that difference is very hard to explain. One of the main sources of competitiveness (and the only that should be most clearly impacted by IT) is productivity.
Are measuring things wrong? Is there some other effect of IT that reduces the productivity to compensate? Is there something else happening that confusingly takes away all the gains from IT? (Perfectly correlated, for 50 years!?) Does plain logic break down due to some mechanism that nobody has found yet?
Nobody is certain about the answer, but theories abound.
Then 2 years later you find out they have separate warehouse gate for each contractor because they never bothered to learn the functionality that lets them assign priorities to contractors. So half the time the warehouse uses like 20% of the warehouse gates for no reason other than lack of training :) Real story BTW.
Or when you consider the alternative.
Filing expense reports certainly sucks, after all.
Allowing everyone in the company to spend money without tracking or justifying it? Yikes.
Archive link just in case that one ever goes away: https://web.archive.org/web/20150213164505/http://www.nichol...
When I try to look up something for work, I am bombarded with distractions.
How does that differ from non-IT processes?
Restaurant staff can also scale like this most likely. Limitations would come from space and available equipment. But this is solvable and why we have McDonalds. etc.
IT is special in having gangs of individual contributors who at some level are like CEOs of the code they create and all the complexity of running a silicon organisation! Every minute is strategy. It is a heavy thinking job.
Investments are not about the money! That is so obsolete thinking!