The Facebook Method of Dealing with Complexity

I've often been thinking about the same thing. I think that most laws should have an expiration date, especially those that were enacted during "emergency" periods (e.g. everything legislated as a response to the Great Recession). The only reasonable exception are the core laws (e.g. the Penal Code), which should be replaced (not amended) by newer versions every so often.

http://www.commongood.org/pages/the-problem

I'm a supporter.

Sort of, e.g. see:

http://www.bbc.co.uk/news/uk-politics-30334812

> Any law that isn't routinely and consistently enforced is invalid.

...since in my view unenforced laws don't do anything except invite corruption.

Prior to each agency's expiration, the Sunset Commission reviews their purpose and effectiveness and then makes a recommendation to the legislature to abolish the organization, continue with modifications, merge the agency under review with another or let it expire.

Agencies subject to sunset review: https://www.sunset.texas.gov/reviews-and-reports/agencies

Since the commission was created ~40 years ago, they've abolished 79 state agencies.

But it does everything for some people or some use cases. I think that's an important distinction.

Take for example the 'Gorilla' issue that Google encountered.

http://nyti.ms/1JlTBab [www.nytimes.com]

Then there was the issue with the bookmarks and suicide categories

http://bit.ly/1JFkmCr [productforums.google.com]

(This one isn't as big of a problem, as the frustration was probably more due to the compulsory usage of the new bookmarks, but it still shows some hazards of auto-categorization).

With social things, it can sometimes be very hard to solve a problem that works for 95% of the population, but doesn't piss off the remaining 5% (often with good justification). Unfortunately, detecting when you're in that situation in the first place is sometimes equivalent to the problem you're trying to solve.

I've worked on some large very heavy business logic heavy systems and often what you find is that the database ends up a dumb store and the logic is entirely handled on the application side, this is fine when the original system only spoke to the one application but becomes a massive problem once you want to allow other applications to access that data store, from experience this is one of the reasons why large enterprise re-writes often fail, the specification on paper vs the massive complexity of edge cases in the application code, years sometimes decades of if/else statements.

Of course the inverse is when the database does have all the business logic and the client is dumb while better that does often mean that you can't get the business logic out easily if the database has become obsolete (some of these systems aren't even SQL based or a dialect of SQL that barely looks like SQL, when you have stuff that was written in the 70's all bets are off).

It's things like this why IBM still sells zSeries with COBOL support for code that was written 30-40 years ago.

Not really sure what the solution is, I'd say some kind of business specific almost domain language that goes beyond SQL or Java (but we tried that, it was COBOL ;) )

As for your suggestion about NoSQL there is merit in that but if you go that route you have all the complexity of the business logic in your application as well as much of the complexity that would have been handled for you in the database via SQL (government systems are mostly amenable to normalisation).

Possibly some kind of SQL/Document hybrid (something like Postgres with JSONB) with a business logic layer over the top might work but I've no idea what that would look like (possibly something like SAP's ABAP which I haven't used but people I know who have don't like it very much), most of our current languages care more about pointers, bytes and function than they do about the actual modelling of the system they represent.

From what I recall the XML corresponded one to one with the paper forms. If you think about how the paper forms work it's pretty straight forward. You go top to bottom entering data and making calculations. Sometimes there are rules (if this line is less than X, goto Y) that you would have to write somehow.

As to how they store it I can only speculate. I imagine they're stored as blobs by return/form. I doubt they would be stored in some denormalized form as it seems unnecessary and difficult. They don't need to query across all people's returns at once. They're probably batch processed return by return.

[1]: https://en.wikipedia.org/wiki/XBRL

I think the example of the Norwegian tax system was a good one, because it demonstrates the article's point without any risk of discrimination. But the earlier example of Gender is a terrible one. Gender is more than just whether you have dangly bits hanging off your pelvis, it's also a very critical part of a person's core identity, and there are a lot of different variations. Lumping everything other than cis-gendered male/female into "its complicated" is a great way to make anyone who doesn't fit into the gender binary feel like they don't belong.

Perhaps a more compelling example would be race. I know there's a fairly standard set of racial categories used when asked to self-report race (e.g. in those optional questions you can fill out when doing various standardized tests), which are fairly broad, and I'm pretty sure the last one is "Other". But at least with race, nobody feels like their racial identity is not recognized as legitimate by a large percentage of the world population (which is a serious problem that non-cisgendered or non-gender-binary people have). But even here, imagine if the test just said "White, Black, or It's Complicated". It should be easy to imagine that a lot of people would be pretty outraged over that.

All that said, if you capture the main categories, and then have a free-form "Other" that people can fill in the details, that's similar in spirit to "It's Complicated" but a lot more palatable. Heck, depending on why you're asking the question, you might not even need to save the answer to Other (e.g. if you're only ever looking at data in aggregate, although even then you might want to pull out keywords from Other in order to report more groupings than the form offers), assuming the people filling in the form have no way of knowing whether you're keeping that data or not. So in the backend you might still have the equivalent of "It's Complicated". But how you present that to the user is important.

---

Edited to add: Looking back at this, I realize that it looks like I'm just talking about whether the user feels fairly represented, which is related to but not the same thing as whether the form actually discriminates against minorities. So on that note, I just want to emphasize the importance of picking an appropriate set of main categories, even if you have an Other, such that very few people should ever resort to using the Other, and those that do should not end up getting unfair treatment as a result. For example, if policy is made based on the demographics of a particular set of people, that policy needs to account for anyone who doesn't fit into the standard groupings. If you just have e.g. "Male, Female, It's Complicated", policy is almost certainly going to be made based on the standard gender binary, without accounting for anyone who doesn't fit.

And on a related note to that, it's not even just the person filling out the form who might feel unfairly represented by a poor choice of categories. Anyone else who does fit into those categories is going to see the categories as a reinforcing of their worldview, i.e. anyone who sees "Male or Female" as a set of gender choices will just be reinforcing the incorrect idea that everyone fits into the gender binary, and this helps cause discrimination.