Hacker News Reader

Comment by kbenson | Hacker News Reader

Bloated beauracracies need agile ways to respond to important situations. Giving them a pass because they are bloated won't make that happen any sooner, and it does need to happen.

View on HN

Organizations like that have no code hygiene. They have smart talented people that are entrenched in their way of doing things. They don't have the money to throw the code away and start over. They don't have the control to enforce code standards. There are a couple of terrible effects that slow them down.

I'll bet you a nickel that the json is generated by a stored procedure. In that kind of environment, you can't really run a local version of the system. if you're lucky there's a prod, qa, and development version. the development version is shared by everybody.

What winds up happening is the dev system has lots of in-progress stuff, and i can't release my stuff till the other in-progress stuff is ready to go, or is rolled back.

Also high priority stuff tends to go to "that one wizard guy". Unfortunately that one wizard guy is backed up with 6 months of other projects, cause he's the wizard.

Is it "right"? no, of course not. These systems evolve from people making good decisions in the moment, that don't really take into account the global state of the system.

Finally, every few years a new CIO comes into power and wants to clean things up. a few new folks buy in, but the older entrenched interests just pay lip service, because they know the only way to actually get software out the door is to do it their way. (They tried and were burned by at least one of the prior CIO approaches)

I think those organizations are pretty screwed. they are incapable of change at the layer they need. Banks, schools, airlines, machine shops, anyplace there's a large sized in house dev team (30+), that team is going to very likely kind of suck. There are exceptions, but generally, it's a rough state.

So, yes, i agree, but actually solving that problem in a way that won't kill the business is incredibly hard.

When these problems affect the bottom line in obvious ways, they get fixed quick. We shouldn't give them a pass just because they are large because that one wizard guy is required, and security problems need to go to the top of his pile, not somewhere in the middle or at the end.

I'm not saying these companies need to magically become less bloated, but they do need fix security problems with the urgency they require. If that means that we all need to make a stink so it starts actually seeming important to them, then so be it. Another way to look at this is that they've reaped the benefit of having a web presence for years, but haven't had to pay some of the associated costs (since they apparently don't have the internal structure in place to review and/or fix these problems). In that respect, they've been playing the odds for a long time and come out ahead (wittingly or not), but that doesn't mean they don't need to pay up when it bites them in the ass.

I've worked where software was not the primary product and was treated as merely another line item on the BOM. They grudgingly had to "do some software" order to make money with their real product. If United is this type of place, here are a few things I'd be willing to bet are true:

1. They don't have a formal QA process or an independent QA team.

2. They don't have source control (or it's very rudimentary, like storing ZIP backups of source code on a network server).

3. Their issue tracking system is an excel sheet on a shared drive.

4. 75%+ of the code was written by someone not currently working for the company. 10%+ of the code is considered "untouchable" because it works and nobody remaining knows how it works.

5. Compiling the final executable takes more than 1 manual step, resulting in screens full of compiler warnings.

6. Management is resistant to any effort to fix any of the above because software is a COST and not an investment.

(EDITed because I can't seem to count to 6)

yes, absolutely. 2 freaked me out the most. I managed to get enterprise github installed, taught people how to use it, set up the client on their computers, pointed them at tutorials, and pestered them regularly. For all that effort, over the 6 months the adoption rate was pretty much just 1 developer in one department - me.

It's just so damn hard to get people to change.

Yes. Any change from the norm can be pointed to add a reason for failure/delay/cost overrun. There needs to be management buy-in and possibly even requirements for projects to reverse this inertia.

Which is why there needs to be pressure on management.

The mentality is: typing in code = assembling the widget, while setting up source control and an issue tracker = decorating the assembly line.

I'm in the position of trying to push towards changes just like these- introducing better issue management, team communication. Anyone aware of useful resources on rolling out institutional changes like that?

Yes. See "The Phoenix Project" (book)

Basically it is all the result of aa long series of shortsighted "business decisions" based on simply doing whatever costs the least or brings the least short-term risk at the given time.

Reply on news.ycombinator.com