Why Is Payroll Hard - Ron Jeffries
c2.com
c2.com
Instead of complaining about the structure or motivation of business rules, excellent developers ask different questions:
- How can I design my architecture to accommodate just about anything my customer throws at me?
- How can I use all the tools in my tool box (including CS and math concepts) to clarify the apparently nebulous?
- How can I use things like Boolean algebra, parameter driven program logic, and data base normalization to provide structure to that which appears to be unstructured?
- How can I use the customer's seemingly unreasonable demands to differentiate myself from mortal programmers?
- What do I need to become in order to say "Yes" to my customer whenever I want?
- How can I use all of the above to actually encourage my customer to attempt the "undoable" in order to differentiate himself from his competitors?
I too love problems like this. A properly designed system like this one will account for every possible oddball situation you can come up with -- and if it can't support an especially weird solution out-of-the-box, it will be easily extensible such that it can.
Too many people code to the goal and don't try to model the underlying logical concepts.
Long story short, I agree with what you're saying, but you make it sound like it's really simple.
It's not.
Creating a single payroll solution to handle one company is very easy. Creating a single payroll solution to handle many companies is also quite doable.
But once you get to the scale of ADP, it's impossible to handle everyone elegantly. Between union contracts and non-standard practices, there are (what felt like) an infinite number of things a customer can throw at you, with one rule wackier than the next.
It's damn hard to write a generic payroll solution that handles most companies. Damn hard.
A) A set of tools to talk with external systems B) A set of tools for manipulating the internal representation of payroll data C) A GUI for interacting with / managing the system.
When their internal process is simple managing the payroll system should be simple. When their internal process is complex you get fat consulting checks. Which brings up a great point, it seems like large companies that simplify their internal systems could avoid writing a lot of consulting checks, so I suspect they must be out there and you just don't hear about.
1) There is "one" way of representing payroll data. There's not. When you get a none trivial amount of clients, all their edge cases put together destroy any ability to fit the data into one conceptual model elegantly.
2) Companies have control over payroll rules. Large companies often can't simplify their systems due to contracts or the subsequent employee mutiny. Unless you want to get sued, lose the lawsuit, and pay out the nose (which I've seen happen), you can't just change how you do payroll on a dime.
Not to mention, there's a ton of disfunction too. Payroll managers aren't always the brightest tools in the shed. One project I worked on lead to the payroll manager getting fired over "bad stuff" I uncovered. And then as I uncovered more stuff his replacement was let go too.
...payroll can be a total clusterfuck. Avoid at all cost.
2) I would argue that much of this complexity shows up because many companies negotiate while ignoring the cost of adding complexity. Company wants to minimize cost, workers want to maximize benefits, complexity is a dead weight that benefits nobody. Once things have been negotiated it's hard to change, but during negotiation they is no need to end up with yet another edge case.
PS: I don't think organizations end up spending 100+ million on their payroll systems because it's easy. I just think being proactive lets some companies minimize those costs. EX: Is Google's payroll system ugly?
That said, there are ways to develop payroll software that keeps extension possibilities and accommodate the variations without disrupting the engine.
I generally agree with the point about designing your architecture to accommodate whatever gets thrown at you, but doing that is hard. Immensely hard. Nothing is concrete, everything is some extensible meta-problem, where instead of writing code to do X, you have to think "how can I make it so my customers can do X, but can also change it to do Y instead." And of course, they want it to be easy to make it do Y instead. Do you just make them write custom code? Do you enumerate all possible options in a declarative fashion and let them choose between them? It's never easy.
The bits about finding structure in what's unstructured, though, I find to often be exactly the wrong mentality, though it's usually the mindset all engineers start with. The best approach in my experience is usually to just embrace the fact that everything is arbitrary, and recognize that you're just going to be writing a lot of if statements (or giving your customers ways to write if statements); otherwise, you'll inevitably A) run into some situation your nicely-structured algorithms can't handle and B) be endlessly frustrated by the exceptions. The processes being modeled by enterprise software are fundamentally irrational, illogical, inconsistent, and arbitrary. Accepting that, and building systems that can handle that, are hard things for engineers to do, since the tendency is to always try to find patterns and order and simpler, more general algorithms.
Not all problems in enterprise software fall into that category, so it's critical to be able to identify which sort of problem you have: is it one that's structured and makes sense? Then write code accordingly. Is it modeling some pre-existing human process? Then expect that the crazy exceptions you've heard about so far only comprise 1% of all the crazy exceptions out there, that you don't even know how to categorize what those exceptions could be, that there's no real underlying order, and that your job is to write the system in such a way that it can handle the fact that the process is arbitrary.
For some value of excellent. Simple, general rules, that are well understood, are better than complexity, even complexity your excellent developer can manage.
How about a developer, who says no to their customer, and comes up with something simpler, and better.
I just submitted this:
http://news.ycombinator.com/item?id=3245550
And declare that progress depends on the unreasonable person.
Are you paving a goat track?
Poor attempts at this will result in very complex systems with unnecessarily many layers of indirection which still can't accommodate to the most peculiar business requirements thrown at them. It's like having the worst of both worlds, overengineering and dirty hacks.
refactoring
version control
DRY applied to laws and contracts
diff tools for laws and contracts
standard interfaces
As an IT professional, framing the right perspective is essential to getting acceptance for better systems. Presenting a more powerful, but simpler, system as "losing some rarely used and inefficient features" is a sure way to get that project rejected. Management will only hear the "losing" part as a loss in total value and therefore worse than the current solution.
Personally, I like to present formalized system upgrades as the price of growth and future growth potential. Most business leaders can wrap their heads around the concept of "we need to focus and streamline in order to reach the next level", regardless of technical experience.
I was trying to suggest that the legal and regulatory framework is what needs to be fixed, not the businesses.
I've never worked at a business that was cut off at the knees by new laws or regs. I'm sure it's happened, but I'm not convinced it's common. What I have witnessed though, many times at many employers, is pure self-inflicted complexity. The did-they-even-consider-the-cost kind of complexity that is nightmarish to implement and hampers future changes.
As an IT pro I can't do much about the laws and regulations, but I can advise and design business systems in ways to avoid complexity traps and help accommodate more sane business processes.
This, I think, is a very healthy attitude; Focus on what you can change rather than railing against things that won't.
There was once a programmer who was attached to the court of the warlord of Wu. The warlord asked the programmer: ``Which is easier to design: an accounting package or an operating system?''
``An operating system,'' replied the programmer.
The warlord uttered an exclamation of disbelief. ``Surely an accounting package is trivial next to the complexity of an operating system,'' he said.
``Not so,'' said the programmer, ``when designing an accounting package, the programmer operates as a mediator between people having different ideas: how it must operate, how its reports must appear, and how it must conform to the tax laws. By contrast, an operating system is not limited by outside appearances. When designing an operating system, the programmer seeks the simplest harmony between machine and ideas. This is why an operating system is easier to design.''
The warlord of Wu nodded and smiled. ``That is all good and well, but which is easier to debug?''
The programmer made no reply.
He devoted an entire session to the French Railroad payroll program. The French Rail- road had gone on an economy program and reduced their number of employees from 540,000 to 360,000. So that was the size of the payroll that was run on the computer in Paris. That payroll must be the most difficult payroll in the world to run. The size of the payroll is only the tip of the iceberg. An engineer’s (i.e. a train driver) salary is a function of the amount of power that he uses. So if he is on an uphill run, he makes more money than on a downhill run. The employees live in government housing and their rent is deducted from their pay. People with a lot of dependents get more money than people with a few dependents. A vet- eran gets more money for each medal that he won in the war. The amount of money depends on the medal. A Purple Heart is worth more than a Sharpshooters medal.
Oy.
Thats why a lot of the technology in healthcare is so far behind. One place I worked at still makes $38 mil/year on a system written in C++ and ASP. Replacing it with a modern stack has been on going process for more than 3 years with no end in site. In the mean time, new installations are being brought up with a whole new set of business rules being added that will just have to get ported later.
Oh, a big item missing from the list is dealing internationally. If you think navigating all the state and local laws are hard wait till you look at another country :)
This sentence is incredibly useful for understanding why modelling a process hammered out by more than three people will always be more complicated than you think it is.
Three? I'd go with one. Hell, I'd go with zero, if the one person is a banker or a lawmaker.
Everything that you do is toward satisfying those requirements. It is hard because each of these pieces are influenced by human interaction. But, that does not mean there are no rules of engagement. e.g. If you have union dues to be deducted, it is a formula to compute it against the total wage you make (or it is a flat deduction amount). If you are paying towards 401k, it is a percentage that you put in towards that.
If you imagine the problem as something to solve in a general sense, you can narrow the problems to fit into models. But, if you treat every requirement as a modification to an excel macro (:-D), then you are in trouble.
Wow, a new person's first comment hits the bulletin board!
You're batting 1.000, gbvb. Welcome aboard.
Except when it isn't. For example, some folks have multiple roles, covered by different unions, with different rules. Those rules are applied to different parts of their "total wage". I suspect that some wages don't count for 401(k). And so on.
None of these things necessarily have simple definitions and they interact in complex ways. Any "general model" is likely to be wrong and seemingly minor changes can have huge consequences.
I never said otherwise.
> It is definitely a lot of work (dont get me wrong).
We agree.
My point is that the models are brittle because every entity (govt, unions, etc) has different definitions for every one of the "common" terms, can change their definitions, and the scope is very context and usage dependent.
New York City, as an example, has something like 15,000 variations for payroll categories. Plus garnishments, domestic relations orders, etc.
How do you compute the extra boot benefit for a fireman who has worked more than 526 hours in a 13 month period?
Or how to you compute the salary of a garbageman who is paid by the ton of garbage hauled, instead of the hour. Except for when they haul ashes, in which case the tonnage is adjusted, or on Sundays, where they get overtime as well. What are the business rules for an ash hauling garbage guy earning overtime on Sunday, which is also a holiday?
The NYC consolidated payroll system, "CityTime"... has cost over $700M since 1996 to build. (That included some serious fraud.) Fraud aside, Accenture was raping & pillaging too -- charging $400/hr for college grad coders.
Treat payroll like an individual bank account with facilities to make and receive payments automatically. Then let actual humans enter the information individually for each person. Add automation of this as an after thought once the problem is fully understood.
When systems of this sort are built, it is usually a combination of Timekeeping (how to measure time spent on the job) and payroll (how to pay for that time). Each section is a big (i.e. millions of line of code and many engineers) worth of effort. All that said, depending on how systems are built, you can a) Create a base system that solves core issues b) Accomodate as much configuration towards that system to allow computation to occur differently based on what the employees do c) Allow customizations of computation when as required d) Make most of the data that can be changed, to be changeable through design (data driven instead of code)
Of course, there will be customers who will want to behave differently because the moon is in a certain position in the system. But, that is when you need to know the domain enough to understand whether what they are asking for is because "that is how they have always done it" or because there is a valid use case that needs one of the b), c) or d) to be updated.
In the past I kept a log of IT project failures as I come across them. There are a few bad payroll system rollouts. http://it-project-failures.blogspot.com/
The Queensland Health Payroll system blew up a political storm because of implementation issues. The cost of the blowout? $219M. http://www.couriermail.com.au/news/queensland/health-pay-bun...
In fact there was an inquiry into what went so badly wrong http://www.insightcp.com/blog/why-sap-hrpayroll-projects-fai...
Payroll systems should never be migrated wholesale. They should be scriptable, for instance pay masters should be able to script the computations to assist them.
Anybody else interested in IT project morbidity could also look at http://delicious.com/search?p=itfailures and http://delicious.com/tag/deathmarch
[1] http://www.nytimes.com/2011/11/01/nyregion/bloomberg-adminis...
I currently develop health care related software here in Brazil. Total chaos. Once I was talking to a doctor, and he said engineers think mostly about the patterns but doctors think mostly about the exceptions. I think the same applies to business. The doctor in question of course liked our system but had a huge list of feature requests (to treat exceptions).
I was yes.
I had a ton of fun doing it... basically you had to write rules to normalize SMTP traps from various vendors, and take action based on them. Network vendors did all sorts of bizarre things -- one ATM would periodically send a "Interface Down" event, with an attribute set that meant that the "Down" event actually meant up!
The hard part was actually doing anything with the events. The regime in charge of my division had a deep seated fear of automatic actions, even actions as simple as opening a trouble ticket.
I hate to be blunt (or worse, snarky) but seriously, everybody does this at least once: modeling and factoring out tons of bullshit when the user is really explaining an ad hoc rules system. Very painful.
At some point during analysis, you have to ask yourself "Do I really want to make the system more complex for each example of X the user gives me? Or just acknowledge that the user is free to come up with all kinds of crazy rules from now until the cows come home, and code that?" (and simplify)
http://news.ycombinator.com/item?id=420650
Of course, procedural code becomes spaghetti code quite easily, but one probably needs the same discipline when doing object oriented/relational modeling...
I've just come off a payroll project that has been 2 and half years in the making, and I can say first hand that all of these things happen. And more.
Acquisitions, benefits, and here in AU award wages for hourly employees make things even more fun. Different sites of the same company can have completely different set of rules and regulations that they adhere to, purely because the unions decided to negotiate differently for a different site.
Ahh fun.