Why Do Enterprise Applications Suck?
michaelnygard.com
michaelnygard.com
Enterprise Applications are built by committee. The committee's job is to put as many bullet points on paper as possible. The more bullet points they've put on paper, the more successful the committee was. The more complicated and long the documentation is, the more work the committee had to do and the more its people should be paid for being able to envision and document a system that is so complicated - without regard for whether that system had to be complicated or not.
Requirements get created for cases that no one knows exist or not. Partly this is because enterprise systems are sometimes thought to be write-once, run for a decade or so rather than systems that will see agile revisions. But partly it's also because, again, it's easier to talk about out of left field problems that will never occur than to actually tackle real problems.
Decisions are made random and arbitrary - but because those decisions are documented, they're fine. So, you want a way of storing that a bank account is closed? Well, we could just have a closed flag on the account when the customer closed it -OR- we could make the account no longer have a relationship to a person, have a zero balance in it and have it be more than 3 months since any activity happened in the account. Yep, that works. Then when a programmer says, but that's not a good way to do that, the question gets asked: why can't it be done that way?
That's the heart of the problem: why can't it be done that way? Of course it can be done that way. It just shouldn't. It's a terrible idea that only an idiotic committee would come up with because the more complicated the solution, clearly the harder the problem (and therefore the more money and adulation they should be paid). If it's simple, they're just doing their work, but if it's hard and requires an overly complicated solution, they deserve praise!
The worst part is that in all of those meetings, you constantly hear "we're building a 90% system" referring, of course, to not tackling outlying cases that may not come up. The problem is that, in the case of enterprise systems, it means tackling every outlying case imaginable while leaving the most common use cases with a terrible interface and sub-par reliability. And the worst part is that most of the outlying cases aren't imaginable so you haven't even solved that.
The problem is what gets rewarded in an enterprise environment and how non-techies see software. Something that is simple, reliable and gets the job done must be solving a really simple problem and took little work, right? In fact, I'm sure many of us have been approached and offered a few hundred dollars to create something just like (MySpace|eBay|Amazon|etc.) because they make the solution seem easy. So, the incentive is not to make something elegant, but to make something that shows exactly how difficult the problem really is - in fact, more difficult than the problem really is.
Likewise, the more features you can write down on a piece of paper, the more impressed your boss' boss will be with what you've been able to accomplish. "Elegant user-interface and code" doesn't take up enough space.
And we're left with crap.
Enterprise Applications do seem to defy reality.
Could a portion of this issue stem from "the market" being left out of the equation? I guess this would only apply to in-house apps.
That said, EA companies seem to do quite well in the market. I guess any committee can pat themselves on the back for finding an "already built" solution that meets (or can be customized to meet) all of their needs. Of course, internal talent must then configure this software monster and build all of those pesky integration pieces to make all of it work.
Well after the purchase, you might hear something along these lines: "Why does it take so long to simply configure something that should work 'out of the box'?"
This is where marketing meets truth (and runs screaming) and bubbles get busted. :)
* No one likes responsibility. When an EA company says, "don't worry, we'll take responsibility," managers like that. For some reason, there is no blame assigned to people who choose bad vendors as long as the vendor made good promises. This is one reason I think open-source adoption is slow. Open-source doesn't promise to work. If an open-source solution fails, it's your fault. If a vendor lied to you, the solution seems to be throwing more money at the vendor. But no matter what, you aren't to blame: the vendor is.
* Most EA software can't be tested nicely before purchase. You get beautiful controlled demos with sales people who can mold the situation. Excuses for why something can't be done in the demo, but work perfectly in real-life or will be coming in a month are common and accepted. Again, they're promising that you don't have to deal with it.
* EA software is expensive. Ridiculously so. Most EA software that we use has annual maintenance fees that rival a salary and still require someone in-house to configure and deal with it. Once you've spent that much money, there is a real drive to build things into it that lock you into it - you want to get your money's worth, don't you? More, there's a huge social-psychological barrier to someone saying, "I'm an idiot that wasted 50 grand and we should really dump what I was paid to choose for something else." Even beyond not wanting to admit you're dumb, you don't want others to know that because if you're not good at choosing platforms, what on earth are you getting paid for? So, the only way to actually switch away from it is to find some excuse about how times have changed and it was good at the time, but not now. Even then, why couldn't you have foreseen that this company wouldn't suffice long-term?
* And finally, by the time enough time has passed to make one of those excuses, you have too much investment in the old system - whether that be training, data, lock-in, etc.
Think of it this way, you shell out $100,000 for a system with $75,000 in annual maintenance/support and sign a two-year contract. You get the system and it's quirky, but you've paid so much for it and are determined to use it. You find out over the next 3-4 months that it's just terrible: the sales people outright lied, you didn't investigate what really needed to be looked at, and it's just terrible for what you need. But you need something now so you pay the company $20,000 (in addition) to build a bit of custom thing. At this point, you definitely can't switch before the 2 years are up (or you've just wasted $270,000 which you will have to explain). Plus, by the 6 month period someone in your company will have found one random thing that they will never let you hear the end of if it ever works slightly differently. It's a terrible "feature" that no logical person would want and would be incredibly difficult to duplicate and so you now have even more resistance from outside IT.
And you can always defend the system as "the cost of doing business" and any shortcomings are the vendor's fault, not your shortcomings (whether or not you should have been better at choosing a system). Markets are powerful, psychology and ego can be more powerful ;-).
Don't forget the fact that your yearly bonus depends on the fact that you delivered this project successfully.
Enterprise applications are a cost to the company, whereas commercial software applications are the product. All enterprise app development must be a give and take between functionality and cost. Businesses will differ in where they fall in that balance, but ultimately companies must reduce the cost as much as possible while maintaining required functionality.
Perfect enterprise applications are therefore by definition the fastest, cheapest software to get the job done. Anything else and the company is wasting money.
I think generally the problem is twofold. First, as the GP pointed out, design decisions (both technical and aesthetic) are made by people who aren't programmers or end users. These people care more about documentation and 'The Plan' than solving the problem elegantly.
The second problem is that most 'technical' people doing in-house enterprise apps aren't that great. I know this sounds callous of me, and there are definitely still great people here and there, but the average ability level is pretty low. My 'enterprise architect' has never even used Java 5 (her excuse: she hasn't had any training for it yet). People like this aren't going to be confident enough to suggest (or even aware of) alternative implementations to those given them by the 'design committee'.
I just started a job doing 'Enterprise Java' (gotta pay the bills until grad school this fall) and they have me working on this project for the next 6-7 months. Once I finally parsed all the buzzword bingo documentation, I realized that what I'm implementing is basically a read-only ORM.
The design as I'm given requires 15+ lines of Java to get a single object back (you know, Post.find(:title => 'foo')) and requires the end user to know which table goes with which DAO goes with which business object, among other things. I'm slowly but surely getting changes approved by my 'architect', and hopefully will, in the end, have something a little more sane.
The fact that they are paying both me and her for six months worth of work to do something that really could be handled by an existing ORM definitely doesn't point to in-house 'Enterprise software' being the cheapest solution. Throw in some of the ridiculous design decisions and 'requirements of our company's standard architecture', and it's not really the fastest either.
As for the Enterprise software from vendors, I'd argue that the problem is that it's solving the problems of these in-house developers and managers, who only have the problems due to over-complexifying (not a word I know) what they do.
Training! Twenty-five years ago, the vice president of the engineering consulting firm where I worked got some bee in her bonnet and announced that the "staff" needed some sort of "training." Being young and politically foolish, I glared at her and said, "Training is for chimps. And don't call a room full of people with engineering degrees staff."
I have no idea what your 'enterprise architect's actual abilities are, but the poor gal talks like a chimp.
The rare examples of big-team success that you say invariably have a big ego (benevolent or not) at the top who has a sense of ownership (Steve Jobs? Linus?).
If everyone on the project has the attitude of "This is MY baby and you will NOT fuck it up," you can build great things. If no one does (the most common state on software teams), you almost certainly cannot.
1. The people who build them are told it's fine that they suck. People who "own the process" (or whatever) don't know anything about software and discount programmer warnings -- "Don't worry, that's not a concern, we aren't here to worry about that." "It doesn't matter if the interface is confusing, because the users will be trained." "As long as there's a workaround, it doesn't matter." "If that button crashes the client, the users will figure out not to click that button."
Sometimes it's just basic stupidity. I know of one case where programmers on a data entry system accidentally implemented a use case that took ten minutes instead of twenty seconds. No problem; the fix was simple. However, they weren't allowed to fix it, because it wasn't "a primary use case." And it wasn't: the users were executing hundreds of use cases per day, and only about five would take ten minutes. So they deployed the software to dozens of hourly workers, who of course ended up spending almost an hour a day, on average, staring at their screen waiting for that use case to complete.
2. Regardless of policy, requirements are almost always gathered from users' managers instead of from the users themselves. Many managers of hourly workers think they know every detail of the jobs under them, but they often don't know the latest details and policy changes. And that's the best-case scenario. A more typical scenario is that a manager comes in with a vague "management overview" idea of what her workers do and a condescending attitude that anything more specific is inessential.
Couple this with the tendency to keep enterprise programmers on a short leash, implementing only the features that were specified in the design process, and you get a result so incomplete that sometimes the only way to avoid rolling back a software release is to remake business processes and policies to fit the software. On the bright side, this can result in simplification and streamlining. On the not-so-bright side, you alienate customers because you suddenly can't serve them the way they've come to expect.
3. The process is single-shot instead of iterative. There's no user testing, and not even any before-the-fact observation of users. Managers don't make any attempt to distinguish between legitimate user complaints and the normal, perennial "OMG something changed" office worker whining. This is especially true when the users are low-paid and not highly educated, which is a shame, because those are the ones who do simple mechanical jobs where their interaction with a software system can materially affect their productivity.
I know of one case where there was a "testing link" from one screen to another that had been accidentally left in the software by a programmer. It turned out to be crucial for workflow, and when it was removed in the second release of the software, productivity plummeted. The workers immediately alerted their manager, but they were ignored. Instead of asking for an immediate bugfix release, the manager waited until several months of metrics piled up demonstrating a massive uniform 20% productivity loss among her workers.
All these point to three problems:
1. The software development process is controlled by people who don't understand software or don't take it seriously.
2. Software developers have no credibility inside the corporation.
3. The corporation always respects the pretense that managers know more (and better) than workers, especially low-paid, uneducated workers, even when this pretense is obviously false.
* The developers don't understand the software they're using. Everyone uses e-mail, so it's pretty easy to know if you're making a crappy e-mail client. I'm not a claims adjuster, so it's much harder for me to understand if I'm building a claims system that makes the most common operations too difficult, or presents the data in a way that doesn't put the most useful information together. Sure, we've got product managers whose job is to figure that out, but they're not really users either, so there's always something lost in translation. Building software that performs a function you yourself perform is just way, way easier to "get right" than building software for a function you yourself don't understand.
* As is pointed out elsewhere, for internal development projects (which a lot of enterprise software is), the economies of scale work against you. It's difficult to build anything robust and functional for under a few million dollars, so most likely you'll end up with something not-so-robust and not-so-functional instead.
* On the flip side, companies make themselves very, very difficult to sell to, which provides a very high barrier to entry for vendors. Part of that is historical: they've been lied to so many times over the years that I can't really blame them. Most of them have a long history of failed implementations and IT projects, and they're naturally risk averse. On top of that, they have all the usual corporate red tape that commons with massive purchases, and since the market for most of that sort of software is often relatively small (compared to consumer markets, at least, or to more general-purpose tools), vendors have to charge a lot for their software if they want to stay in business. Unfortunately for their customers, that means that it's difficult to sell to them, which lowers the number of entrants in the market, which reduces competition on the vendor side, which leads to poorer quality software.
* Enterprise software is often boring to develop, you don't understand the use case, you can't show it to your friends or family or anyone else that will care, the deployment model often sucks, if you're a vendor you have to make things configurable (which makes it 10x as hard to develop, reason about test, etc.), and the companies you're selling to make the sales process incredibly difficult. And that's just working at a software company; obviously corporate IT in-house work is even less appealing. Understandably, the most talented developers often choose to work elsewhere. My impression these days is that the "you can't show it to anyone" bit is the most important for people; if you work for a company people have heard of or on a product people have heard of, you can take pride in showing that off, and not having that ability to show someone the cool stuff you're doing takes away a lot of the fun of software development.
* The small market and difficulty of deployment generally means that real user testing is difficult and costly, so people just don't do it. As a vendor, there's not much we can do (we can and do go watch people using the software, but there's not much we can do before we ship a version), and that's not a core area of expertise for the IT departments that are installing the system (or building one themselves). We can't just throw something out over the web and do A/B testing, or push a new version and roll it back if people report too many bugs.
* The market isn't yet there for hosted enterprise apps, so you're stuck with in-house deployments for now. I've commented on why that is earlier, but it comes down to both pricing, customization requirements on the customer side, integration requirements, and market valuations. It might work for certain types of enterprise applications that have large enough markets to make it scale, but for industry-specific systems where the target market is in the 1000's of companies, the math doesn't really add up.
There's definitely an opportunity there for a company of talented engineers to disrupt some portion of that market (which is what I like to think Guidewire, my employer, has done). But it's not as easy as just building a web app, attracting users, and slowly growing it: the domain-specific knowledge required is off the charts, and the barriers to entry are incredibly high.
I think that is a MASSIVE point which very few people realise, until they go down that path. And then they are grumpy and don't want to talk about it ;)
It really bothers me that all these talented kids are outside of the companies that produce most of our economic activity - so big corps are still using the same crappy unproductive software they were a decade ago.
Not only is nobody inside these companies causing disruption - but I see few startups targetting core business functions like purchasing or logistics.
VCs could be driving this. Instead, like a herd, they are all trying to back the next greentech, cloudtech, socialtech etc.
The biggest opportunity to create cost savings for new businesses is in reinventing the way the enterprise apps work and communicate with other enterprises - for instance creating auction and bidding systems to improve procurement.
A startup has more chance of saving 1% of costs for a Top 500 company - than it has of being the next Facebook.
With apps like Google, facebook, or any number of other startups, building a good product is in line with what your customers desire. Facebook was cleaner and more usable than Myspace, and customers responded in droves. Google produced better results and was easier to use than the competition, and customers responded...
Enterprise software doesn't work like this. You're not selling to random data entry guy #21243 who will actually be using your big fancy enterprise package - you're selling to a PHB who couldn't care less how great and cutting-edge your product is. Someone told him to implement some system, he wrote up a RFP, and here you are. All he wants you to do is jot down a bunch of bullet points, make some big promises, and offer to take responsibility if it all goes down the toilet. He's not interested in your superior usability - there's no demand from up top for more productivity from the data entry monkeys, and it's not as if he will have to deal with this software every day...
There's no room for innovation there, because those who pay for the product do not desire it, and will not reward it.
And it's not true that corporate buyers don't care about quality or innovation - often they simply don't have much choice. And, of course you have to speak the right lingo.
- They lack the best people. People at the bleeding edge of software development and interface design tend to work for companies like Apple or Google, or to start their own thing, not go build a risk-reporting system for a bank.
- They go about development the wrong way round. They start with a huge list of features and then try to make it usable. Most start-ups do the reverse - they start with something usable and only then think about adding features. I'll be writing a blog post about that on http://www.woobius.com/scribbles sometime soon...
Enterprise applications are highly customized systems with deployments of 1 in many cases. We talk about the outrageous cost, but consider an 500,000 dollar application. That's a lot for a company, but that's not a lot for an application budget.
With a 1.6 multiplier for expenses, that's enough for 2 fte senior engineers @ 90k or 3 young ones at 60k, .5 QA @ 70k, 1 manager @ 120k for 20%, a half a technical writer and a half a business analyst - let's call it one person @ 45k. That's 454,400 there. You haven't even accounted for travel yet, or a part time contractor for domain expertise or a graphical designer or someone to design an aesthetically competent interface.
Now, the cost of programmers might be high for senior outside major metros, but the numbers aren't far out of line for a project in less expensive markets.
How much are you going to expect two (or three) programmers to do for a full fledged product? Add to this the shifting requirements, client corporate bureaucracy, and associated complications that have been listed below, it's really no surprise that enterprise products have difficulty with quality.
I've given it a bit of thought on how to deliver better products in the sphere, but I start designing and realize that we've quickly found our way into architecture astronomy quickly. I'm starting to think the best route is actually something akin to http://baseportal.com/ with a rules workflow system and a little more front end polish.
The problem, again, runs into the customization problems. The more customization that is allowed (and that is a requirement in enterprise), the more complex the system becomes and the advantages of quicker development dissolve.
My own business, and many of the YC start-ups, bring products to market for considerably less than $500k. And those products tend to kick ass (well, the YC ones do... I like to think woobius does too).
Perhaps corporations would have better luck by offering $100k each to 5 teams of 2-3 hackers, selecting the winner based on their internal user's feedback after 6 months, and then offering that winner a $200k/year regular income for maintaining the app. (Yeah, I know, I'm dreaming...)
2. You are developing for someone else's design, which often changes midstream, and it has irrational requirements due to data flows from previous products. I worked with a product that had to support the public health information network logical data model (PHIN-LDM), an absolute monstrosity of a database. If we were building the system privately, we could have shaved the effort in half, if not more. Importing previous client data can be monstrous, due to poor system checking and user hacks to make a system do what it was never meant to do.
3. Deployment is more expensive. Gathering clients is more expensive. Each means traveling over to the client to gather requirements and install, and to conduct training.
Now, if you can do all this, there are a slew of contracts ready to happen. Get a team of three together, and subcontract out to integration teams and prime contractors. All you have to do is find a first and second project to gain momentum.
As a startup you face several problems if you want to get in on the enterprise market space. First of all, you're a start-up. You might go bankrupt tomorrow. Secondly, no-one else in the sector you're targeting is using you. Well, that's a no-go right there. This is a chicken-and-egg problem. Thirdly, you probably lack vendor support or recognition from analysts. Oops, you just failed the skimming test (as in 'skimming over basic requirements'). Can your product even execute in the major J2EE stacks? (Probably not).
If you're not 'proven' technology, then you have a hard time moving into this space.
And this space is not driven by user satisfaction or programmer happiness.
Look at how far Google has managed to move into the 'enterprise space'. Not very far.
In the Enterprise, the UI is usually just thrown on as an after-thought. Of course it's going to suck.
http://weblog.raganwald.com/2007/09/we-have-lost-control-of-...
It's pretty soul crushing to spend all of your working time building systems for inventory management, or AP/AR, or whatever mundane (but incredibly important) business function you're working on. This is why people make startups I suppose.
I agree that, if you have the mental toughness for it, enterprise software is a GREAT way to make money. During my days of having a real job, we were selling our software for $250,000 an install.
The way I write (very small) biz software is get a contract from the top, along with the specs for the first version, and over the next years (pretty much forever) get specific requests almost exclusively from middle management.
It's hard to scale though. I do what they ask because I care about my work, but on a strictly contractual basis I could reject or indefinitely postpone 90% of it.
Our protos were quick and simple affairs using jquery. The CTO refused to integrate jquery into the application itself, and rewrote everything custom, which worked half as well as our simple protos and took months longer to develop, had they built the front end with jquery.
The CTO lost his job recently due to cost overruns.
The problem is more about internal applications that have nothing to do with the identity of the company itself. Personnel management, time tracking, accounting, and payroll apps suffer from poor quality if built in-house because they are part of the cost of doing business independent of what that business may be.
Our users were satisfied with RTS's planning results. The UI gave them some trouble. The major dissatisfaction with it was the difficulty in updating/maintaining the many desktops at our company and at our partners' offices. A year ago, word was that UPS was considering possibly offering RTS as a hosted service.
So, is RTS amazing? Well I haven't seen any papers from UPS computer scientists describing their novel contributions to the algorithmic state of the art. The UI is a bit clunky; users don't fall in love with it. But, it doesn't crash often and it does a satisfactory job solving a non-trivial problem, which is more than I can say for typical in-house enterprise software.
Size. Too damn many people involved. Startups and small companies innovate/code circles around Enterprises cause they are small 2-5 maybe 10 people per project. They all know each other, they're all responsible, they're all committed to success and there's no one in their way.
The more people involved the closer to (their)average you get. It's possible to have small number of awesome, above avg people, but not the 50-100+ involved in enterprise app, not consistently.
If enterprise's gave cross-functional teams of 5-10, 20 people autonomy and a general goal they would rock out.
I remember one company, the CEO was trying to prepare it to be bought by a large company - one bullet item - "enterprise contact management and sales tracking software needs to be in place" . So SalesLogix was bought, 15 people were trained on it - one week later no one was using it.
The top salesman was using his (original version, this was late 90s) PalmPilot and his Windows laptop instead.
The sales engineer that had been with the company the longest and was employee #5, had 3 big accounts to keep happy and maybe 2 people to talk to at each account, for a total of 6.
Conversely, a lot of open source apps suck too. A lot of them also don't.
Why are we stereotyping again?