It Takes 6 Days to Change 1 Line of Code | Hacker News Reader

It Takes 6 Days to Change 1 Line of Code

edweissman.com

457 pointsjacobr231 comments

Honestly, for code that controls millions of dollars worth of business and could seriously screw over a lot of people if a subtle flaw was introduced, I don't see a problem with doing it this way.

Might be overkill for one line of code, but where do you set the threshold? If you say any change longer than four lines is subject to this process, then people will work on trickling in new stuff in increments of four lines at a time.

Also, I can do a lot of damage in one line.

"Changing one line of code took 6 days!" is a pretty common WTF story. It comes up every so often, and we all get a chuckle at the ponderous rate of change in large organizations.

However, "Changing one line of code broke everything for 6 days!" is also a common WTF story. As a rule of thumb, you can't avoid both of them, because avoiding one entails the other. I wouldn't fault a company that chooses the first story over the second.

The problem is that things that scale up don't always scale down. The processes implied aren't that bad, for handling large commits to a large process. They're obviously overkill for small changes. But just because the processes are a negative effect on this change in particular does not mean that the processes are a net negative overall. More flexible processes could also end up being more trouble than they're worth, due to allowing (encouraging) sloppy work.

Or you could just hire responsible people and ask them to use their judgement, you don't have to choose one or the other you can instead apply judgement as to whether this is going to screw shit up or not.

Honestly, though the best way to handle this situation is with a phone call to the CEO/IT Director getting him to write you an email allowing you to override all the checks to get this in ASAP.

Once you're known for clearing things with hire ups, QA will start being reasonable. You have to keep in mind that most people in QA are type C personalities and will be a stickler for inane rules.

http://www.buzzle.com/articles/type-c-personality.html

Feh.

For starters, "just hire responsible people" can't be applied retroactively. Not to mention that… most people are responsible, or rather, the road to hell is paved with good intentions.

Secondly, it's an incredibly difficult thing to scale. That works great in startups, but by the time you have things like "HR Directors" you're no longer capable of having insight into the rest of the organization.

There is a lot to be said about how bureaucracies are the formalization of common sense, and a lot more to be said being able to scale processes out such that you don't end up in situations where stakeholders have no visibility into the work performed.

I just feel that the answer is more complicated than "hire A players".

> Or you could just hire responsible people and ask them to use their judgement

If you know a fail-safe way of only hiring responsible people who always have good judgment... then I hope you're being paid millions of dollars in consulting fees, because nobody has really figured that one out in a general way yet!! :)

Also, those people tend to be expensive... sometimes it's far more cost-efficient to hire less-perfect people, but put rules in place to minimize any catastrophic damage they can create, at the expense of efficiency.

Following the rules to the letter also produces bad outcomes.

The issue is not that using judgement is perfect it's that using judgement usually produces a better outcome than blindly following a rule.

Yes, rules should be put in place for the most extraordinary circumstances or places that good judgement is known to fail. The rule has to prevent the bad thing to be effective while also not creating something worse.

In this case, there was someone with basically good judgement, but they were operating from high within their hierarchical framework, which inherently induces delays. In retrospect, Philip, David, or both, should have been copied on the ticket and directly overseen the policy violation that they deliberately set in motion.

Simply ordering policy violations from the throne doesn't work in practice. Either the person who made the decision or a trusted lieutenant who understands what's going on has to see it through to the end, otherwise the bureaucracy in the middle will simply continue on its preprogrammed course.

I think the reason this fails to happen(philip/davic overseeing the ticket till production) is because of this (Heads I win Tails you lose)principle http://www.ribbonfarm.com/2011/10/14/the-gervais-principle-v...

> Or you could just hire responsible people and ask them to use their judgement,

Or, just as easily, you could sprinkle magic fairy dust over your servers to solve provlems.

Just because they're smart doesn't mean they don't make mistakes. If you allow one line changes to pass without review, eventually someone's going to make a breaking one-line change.

Did you miss this part:

> It [sic] we don't do this right away, we'll have to have a layoff.

As a company, it's great to have a process for incremental changes, but you also need to have a process for critical hot-fixes.

You're a phone company, and adding a new feature should take months. I get that.

You're a phone company, and because of an unforeseen Cinderella problem, 10% of your customers can't make phone calls. At all.

It is unacceptable to have a one-size-fits-all process, no matter what your business is.

> As a company, it's great to have a process for incremental changes, but you also need to have a process for critical hot-fixes.

And as soon as you do, everyone thinks that their problem is a good candidate for the critical hot-fix path. At the very least, you now have to sink minutes per day into arguing them down from using it. Worth it? Possibly. But not always.

If you have a single (actually) business critical incident that cannot be resolved, the company will fail. It's called risk management. You spend minutes per day arguing with idiots to not have the company fail in an emergency.

You need a good operations team with the authority and willingness to say "no" with overrides coming only from senior management, the authority to say "yes" for the obvious stuff, and a direct line to senior management for grey areas they're not comfortable making a call on by themselves.

The operations team will spend a few months saying "no" a lot and justifying their decisions to management. Eventually it will slow to a trickle except for a few really stupid people who lack reading comprehension and any sense of pattern detection. Management will eventually get tired of it and forbid the stupid ones from making hot-fix requests.

Agree completely.

Step One is "Act Intelligently."

No process is going to help you skip Step One.

What's a Cinderella problem? I'm assuming it's a reference to the midnight deadline.

The code behaves differently after some event happens.

Midnight. Y2K. Daylight Savings Time change. Leap Day. Congress changes when Daylight Savings Time happens and one component in the system didn't know about it. Year 2038 problem.

2038 is going to be a fun one, isn't it? :)

Dr. Peter Venkman: This city is headed for a disaster of biblical proportions.

Mayor: What do you mean, "biblical"?

Dr Ray Stantz: What he means is Old Testament, Mr. Mayor, real wrath of God type stuff.

Dr. Peter Venkman: Exactly.

Dr Ray Stantz: Fire and brimstone coming down from the skies! Rivers and seas boiling!

Dr. Egon Spengler: Forty years of darkness! Earthquakes, volcanoes...

Winston Zeddemore: The dead rising from the grave!

Dr. Peter Venkman: Human sacrifice, dogs and cats living together... mass hysteria!

Mayor: All right, all right! I get the point!

Funny quote. I love that dogs and cats living together came after the dead rising from the grave. Not sure I'd put them in this order. :)

ICL George let operators flick a switch to make the current day have 60 hours until the switch was flicked back.

Reliability and quality are obviously important, but you have to question whether all these steps are actually helping you to achieve that.

All of these items in the story add overhead and reduce agility, but potentially do very little to improve quality:

  - Mandatory paper trail via fields on the issue tracker
  - The need for documented internal sign offs 
  - Standards and policies with no clear reason for the existence
  - 'Micro' Code reviews
  - Excessive permissions and security processes internally
  - No pragmatism with regards to testing the change

This kind of stuff is a constant and substantial overhead on getting anything done. You need to look at these processes very critically and be very sure of the benefits in terms of quality before introducing it.

- Code reviews

I have to take issue with this one. Code (and Requirements, and Design) reviews, when done properly probably have the greatest impact in improving product quality. No, I don't have time to look up the SEI research on the subject, but it has been consistently shown to be true.

Part of the problem I had with the article was that Shirley was rejecting code for non-specific reasons. You can't say "it's not written down anywhere." If it's important enough to reject the code, it's important enough to write down.

I like code reviews for higher level concerns - new class hierarchies, big pieces of refactoring, new areas of functionality etc.

However, I don't see as much value in reviewing low level changes such as one liners or odd bug fixes.

Making code reviews mandatory is too much overhead and too inflexible for me.

We have mandatory code reviews for all changes and I think it works quite well. A person making a one or two line bugfix may think it's quite innocuous but it may have a bigger impact on other parts of the system than they realize. Many eyes help reduce the likelihood of that.

I dunno... the odd bug fix has the potential to really screw things up, when the person fixing it isn't the person who wrote it originally, or it's been months since then.

Mozilla and Google require code reviews of all code changes.

Here are some notes about Firefox's code review process:

https://wiki.mozilla.org/Firefox/Code_Review

https://wiki.mozilla.org/User:GavinSharp/Code_Review

A few notes on those points:

1) The fields are probably mandatory because so many times routine entries were not filled correctly, and even MORE overhead was required going back and forth between teams to find out all of the details. If there is a way to avoid this overhead, again, how do you stop people abusing it?

2 & 3) Yeah.. bad overhead.

4) A sanity check is ALWAYS a good idea, whether its a fresh intern or the most seasoned programming in the company making a change.

5) They security processes might seem excessive but once again are probably in place due to issues in the past that have required the review/signoff be there to stop major disruptions getting through.

6) As someone who worked in QA for a while, there is no way in a hell someone can say "push this change through" without a strong questioning of why. QA is there (in most cases) to be that last line of defence. Often letting people know just how many people are really going to be affected by this change.

Being on a testing team often gives you a broader oversight of how all of the components of the code work together, while people working on one project can sometimes get tunnel vision on getting their thing out the door.

Edit: Just like to point out, I'm agreeing with you

I really like to see a pragmatic relationship between Dev and QA where you reach agreement on the best way to build confidence in a change.

QA should absolutely be able to push back on Dev with regards to quality and relevant test cases, but likewise Dev often need to be able to direct the testing and mutually agree on the scope that will get the change signed off adequately.

Exactly. The point bengl3rt was making, I assume, is that cavalier avoidance of process is a bad thing because it allows mistakes to happen that would be caught. That much is true.

But the assumption that's wrong is that all processes avoid mistakes. Clearly they don't.

And some of the examples here are just plain cargo cult misapplications of good ideas. The point behind code review is to catch design flaws in new code. You never want to demand refactoring of existing code in order to patch features, that hurts, it doesn't help.

This quote is appropriate here:

Skilled people without a process will always find a way to get things done. Skill begets process. But process doesn’t beget skill. Following a recipe won’t make you a great chef – it just means you can make a competent bolognese. Great chefs don’t need cookery books. They know their medium and their ingredients so well that they can find excellent combinations as they go. The recipe becomes a natural by-product of their work.

From http://the-pastry-box-project.net/cennydd-bowles/2012-march-...

Er... great chefs do need cookery books. They may not refer to them as often, but you won't find many chefs out there without a collection of cookery books. They still use recipes for things they're not familiar with.

Besides, skill may beget product, but it doesn't necessarily beget process.

>> Besides, skill may beget product, but it doesn't necessarily beget process.

Oh boy, talk about not seeing the forest for the trees. For skill to beget a successful product, you must follow a process to get from nothing to product. So in the course of creating a successful product you have automatically created a process -- the process to build the product. That process may be used one time, or multiple times -- but it's still a process.

Not seeing the forest for the trees? Hell, if we're going to use that loose a definition, everything is a process, regardless of skill level. Even if you end up without a product, to get the steaming pile of crap you abandoned, you went through a process.

Demand, no, but often it is the best idea.

Having just (as in hours ago) been through the wringer of adding new features to a mass of spaghetti that had not been touched in 10 years, the only way I could maintain my sanity (and have some assurance beyond regression testing) that I wasn't breaking something, was to refactor the existing code and then insert my changes.

Sure. That's the point behind refactoring, it makes changes to existing code flow more smoothly. But the case here doesn't fit that at all: the change as described was a change in configuration (that just happened to be stored in a code variable), yet it was being reviewed as if it were a new feature being added through development. That's the "cargo cult" part -- refactoring in the course of development is good. Rules demanding refactoring are bad, because they hit false positives (in this case, a high priority configuration change).

And more importantly: a complete lack of trust within the organisation.

Agreed. 19 times out of 20 people will do the right thing.

Therefore, process like this unnecessarily slows down the organisation 95% of the time.

Much better to trust people and implement very specific and very carefully designed safety nets such as fast rollbacks.

In most cases, the benefits of just trusting people will massively outway the costs of the occasional slip up.

If these are the proportions then I really disagree. And actually these are the proportions I see (or better). If you have 19 out of 20 changes done correctly, that means probably 1 every week will fail. Now the question is how will it fail - spelling mistake? single request rejected? single request failed? customer data lost?

If you let too many simple issues through, you're likely to find yourself completely failing when a very simple thing breaks. First you'll find that your logging is not correct, so you'll need to fix that first and reproduce, then you'll find that the rollback doesn't quite work and you've got some bad data to fix manually, then you'll find that this is actually a simple error masking some really nasty bug, etc. etc. I've seen that once or twice and I really believe in the broken windows theory now. I'd be more glad with a reasonable slowdown, than people pushing ahead instead of stopping to think about long-term issues.

Yes, I'm the guy who rejects ~4 out of 5 changes during code review initially (on average). Then again, I'm the guy who gets woken up when things fail - not everyone does, but it really gives you the appreciation of why you want to test those exception branches. I hope that people will be as strict about stuff I submit too.

> Also, I can do a lot of damage in one line.

Especially if it takes a week to revert it. You reap what you sow.

I'm glad this is being said.

I would add that crappy processes and tools are usually in place because the management/leadership in an organization has made them too brittle. While you don't want to wild-west every "critical" issue, bad spots in a process will naturally highlight themselves over time.

Let your tools and processes constantly evolve instead of blindly subscribing to a methodology, and you'll be in much better shape.

If you say any change longer than four lines is subject to this process, then people will work on trickling in new stuff in increments of four lines at a time

With proper test automation and continuous deployment, trickling new stuff in increments of four lines at a time is the right way to do it.

I agree that it's worth being careful on mission-critical code, but this isn't being careful, it's just box-ticking. Ed said he'd tested the actual change, but most of the time is taken up by shifting it out to some parameters file, lengthening its name and waiting for permissions.

No one seems to have pointed this out yet, but the code that runs your manufacturing line is indeed "mission critical". Furthermore it's likely that your customers demand that you have strict processes in place for changes, and can actually audit you at their pleasure. So being so bureaucratic about it is not surprising.

The Web 2.0 world works very differently, of course.

To clarify, I'm not suggesting that it's not mission critical, just that I don't believe that that kind of process is actually making things any safer.

Indeed. But that doesn't mean that you have to like taking 6 days to write one line of code.

That said, 6 days isn't so bad as long as we're talking about the time it takes to write the code and put it into production. It's another thing to take 6 days to put the code into source control.

In many cases it's trivial to rollback. Why would you fight change if changing the change is as easy?

In this case process overrode rationale; and was cheered as a result on its own.

Agreed. I think the main problem in this story however is the the red-tape between people and not so much the process itself.

Ask Homer, well Homer is not in so ask Marge, well I don't have access to Marge. Delays like this that come from artificial communication barriers can be easily avoided if they worked on improving communication throughout the organization. It would probably have saved them 1-2 days, if not more.

As mentioned above, Homer and Marge are servers rather than people.

I seriously don't see the problem. The author seems to be assuming that taking 6 days to have a change made is objectively bad.

Does the company run? Does it make money? Do the systems work and stuff?

Well, if whatever you're doing now to keep the systems working and the company running and making money obviously works so I'd say it's objectively good.

The experience may feel bad subjectively but how you feel is of trivial importance when compared with the health of the company as a whole.

It's all about risk right? Like if the company wants to spend $50,000 to change 1 line of code, and it's not going under, then that's obviously the right decision. It's just the same as spending millions of dollars per year on insurance and then complaining when the place doesn't burn to the ground.

The "move fast and break things" attitude of Facebook works for Facebook - it's a different company.

Small startups can break whatever the hell they want because they're not making any money yet.

An established company with a functional system that makes money has to take measures to protect it, because that functional system is where all the value is in the company.

I don't see the problem either.

The article is extremely self-centred, and doesn't deconstruct why this is bad for their business.

The programmer's job is to help the business achieve its ends. If she feels stymied by process she needs to discuss it with the business. If she's criticised for taking too long she needs to lay out where the time went.

Imagine if lawyers wrote articles about how it took months to convict someone that's obviously guilty?

The business wanted to s/3/4/ on a single line. The reality is that President, CEO, CIO, etc. don't really care about whether this parameter is configurable, if there are specific test plans, if the test team is happy with the variable name, etc. They don't want to do a layoff. So they asked the developer to perform the s/3/4/.

The developer did so, reasonably. Then it sounds like the self-righteous testing and security people stepped in and basically said "well, this might be delayed because it doesn't follow our standards and we don't have all the sign-offs. This might mean layoffs, lost profits and so on, but damn it we have to follow our policies!"

Sounds like the IT dept. is not enabling the business. They are, instead, hijacking it, substituting their own priorities for those of the whole enterprise.

IME process doesn't spring fully-formed from the head of Zeus.

You get test plans when somebody goes cowboy on live code and breaks tangentially related processes. You get security reviews when somebody goes cowboy on live code, everything looks great, and then a month down the line you lose days of work when someone changes a password. You get mandates to create parameter files when someone gets burned having to run through this entire process over and over again when business rule changes back and forth.

The reality is that once this change is done the proper way and the business parameter is moved to a file, the company will never lose time over this issue again. It will be configurable just as fast and responsive as we all feel it should be, without any concerns about breaking things down the line.

Is it frustrating? Sure. But that doesn't mean it's wrong.

Going cowboy on live code, even for something as trivial as changing a 3 to a 4, can and has created large problems.

Agreed. Process comes from a need. My point is that it sounded like the test and sec. people were not trying to speed it up. They seemed to offer little help or alternatives. Perhaps the communication process broke down. Did Ed know the importance of this change? Test and sec. certainly did not seem to.

In the enterprise, it's always an emergency. If it isn't, it gets ignored, because the pipeline for getting 'non emergencies' done is constantly pre-empted for emergencies.

That said, every good process has the ability to handle exceptions. If it was truly the emergency it sounds like, he should have been able to make the change to get it up and running ASAP and then made the change properly through the normal process.

But the question of whether an "emergency" truly is, is a complicated one.

He may have changed only a single line, but making the queue 4 months instead of 3 could break other parts of the system. Perhaps the "save data" and "backup" functions assume 3 as well, so you'd lose data. Perhaps other assumptions are made in the code, so with a 4 you'll end up with buffer overflows. Those test and security people are in the process to catch these issues before there are problems. It may seem a trivial change, but people make assumptions all the time.

"Imagine if lawyers wrote articles about how it took months to convict someone that's obviously guilty?"

If they don't, they should :)

(and no, I'm not kidding. Justice would be better served by better judicial processes)

Indeed,

Also, it sounds like it didn't take six days to change the lines. It sounds like it took maybe two hours spread over six days. Not being able to pivot immediately might be a problem but it sound like it would the president's problem, not Ed's problem. But it might be anyone's problem. It just takes a while to do stuff when you're multi-tasking.

Did you know the computer also wait many instructions before it performs the one you asked for, even in "c"?

Besides that, most of the delay in this story was not caused by process but by human error and sloppy work by the OP.

There are some hints of organizational dysfunctions along the way, but those were not what caused the major delays.

Without those screw ups, it would have been 2 to 3 days from decision to production. That's pretty decent for any company larger than a 10 people start up.

I generally agree that this process doesn't sound all that terrible for code that controls a large, live manufacturing line. Try changing the code that runs a pacemaker - you'll learn all sorts of things about process.

This line however is the road to hell:

"if the company wants to spend $50,000 to change 1 line of code, and it's not going under, then that's obviously the right decision"

Just because a company is successful does not mean it does everything right and does not contain the seed of its own destruction.

Really, I think the problem with this scenario is not that the company wanted to "spend $50,000 to change 1 line of code." The problem is that management wanted to spend $500 to change one line of code, the dev team spun up to spend $50,000, ended up spending maybe $5,000 before management told them "no really, we don't care about what you think, give us the $500 fix."

If you want a change request processed quickly, you should get someone in upper management to help you herd it through the system. Literally, take your VIP with you, and walk up to the desk of each admin in turn. Say, "Don't wait for Tuesday's meeting--approve this right now." And then walk to the next admin and do it again. And then walk to the test guys and say, "Drop everything and test this now." And then gather up the six essential stakeholders for the CCB meeting and say, "Impromptu CCB meeting, right now in the hallway instead of next month. Approve this."

I've gotten software changes that usually take two months to trickle through the system through in a day. You have to spend all day doing it, though, and so does your VIP.

But if it's really that important . . .

The tricky bit is that if there is a bug in the change, the VIP might suddenly be busy with other work.

And as you said, a smart VIP is there in the room, suggesting things get done. Explicitly not writing things down in emails.

That works once. Then the next time you go to each desk, the person there has already checked with their boss, and says "my boss says you need to go through the correct channels and if you don't like it, you need to talk to him."

That's why you have a VIP with you. If he's not her boss's boss, the whole thing is a lot less likely to work.

Wishful thinking. Each of those people has a thousand other things to do than to hold your hand all day long. Only works for things that are super-duper-important.

Additionally.. Going to "tell" on your buddies all the time and holding "the grownups" over them is terrible whinery. Everyone on the playground hates kids who do that.

People don't realize processes, reviews, standards, and audits are there because of painful (costly) incidents and breakdowns in the past. Changing one line of code in the production app is fast and easy. Crashing the system or screwing up the app logic can cause huge disruption to the operation.

After every single crisis, managers got together and vowed to never let this happen again, and thus processes and reviews are born. After couple years, things will get so stifled to the point of what OP was experiencing.

I liked this post on the topic - http://37signals.com/svn/archives2/dont_scar_on_the_first_cu...

"embed the learning in the organizational memory as a story instead of a policy"

That memory is hard to maintain with procedural changes in modern times, as employees move around quite frequently.

That "legacy code must be brought up to current standards whenever it's touched" policy doesn't seem very wise. I'm all for practical refactoring, but it's a major decision to be treated with care when it comes to legacy code. You don't always want to invite it. I certainly wouldn't want to enforce it.

I mean, it's one thing when your oldest legacy code is two years old, and it's all one program. It's something else entirely when it's, for example, written in Fortran and running on hardware you can't buy anymore. Sometimes you don't want to rock the boat any more than you absolutely have to.

Yes, for an emergency change (change or there will be layoffs), this is a rather dangerous policy. The goal of any emergency fix should be "do as little as possible to solve problem".

I get the feeling that too many people in the chain did not treat this as an emergency. It seems very business as usual. When a company is talking layoffs, it is an emergency.

Yeah, I agree. The first time he told someone "Philip wants this done quickly" and they said, "Sorry, it's policy," the next move -- at least my next move -- would have been to walk to Philip's office and say, "Will you come tell this lady that this change is more important than that policy?"

I mean, they don't have a way to know it's really an emergency. Every engineer thinks everything's an emergency. There's nothing like the guy in charge standing there while they work to communicate THIS IS ACTUALLY AN EMERGENCY.

This is similar to laws that require buildings be brought entirely up to code any time any renovations are done. The intent is to keep buildings up to code, but one of the nasty side effects is that buildings which are already substandard continue to deteriorate because of the massive expense (in time and effort as well as money) that is required. I've seen half a dozen instances with churches and other nonprofits deciding against simple renovations (like adding a wheelchair ramp) because they didn't want to have to redo the entire building's wiring, widen every doorway, and put in an elevator.

If you require a massive overhaul to be tied to a minor fix, one likely outcome is that the minor fix simply doesn't get done.

If you require a massive overhaul to be tied to a minor fix, then you have let the thing rot for too long.

In your example, those buildings that deteriorate will at some point stop being liveable, and lose in value, which will finally mean that the proprietor's failed business approach will become weeded out. I, personally, welcome this sort of development.

"Being out of code" isn't always a result of deterioration or neglect; sometimes it's just a matter of the code changing. Structures built in the 1950s may not have, for example, ADA-compliant accessible restrooms, though the restrooms may be perfectly usable. But putting in something as simple as a wheelchair ramp at the front door might require also remodeling every single restroom in the building to be ADA-compliant.

I've seen half a dozen small religious groups go through this sort of problem -- because they can't afford to make the church/synagogue/etc completely accessible, they aren't able to make incremental improvements. The buildings remain perfectly functional and well maintained and retain all of their value, except in the eyes of the one guy in a wheelchair who can't get a ramp put in because his church can't afford to drop 15 grand on all the other not-very-useful-but-required stuff.

Relating this back to code: if you have a policy that requires any old code must be brought up to new standards any time it's touched, then there are at least some occasions in which someone could have made a useful incremental improvement, but is stopped by a stupid policy decision that would saddle them with burdensome work. The net result is that small, high-value fixes get put off because of large, low-value busy work.

I agree in theory, but it's a slippery slope...

If you make one exception, then a month later, you discover that nobody is ever bringing old code up-to-date ever.

And that is how creativity dies. Not all at once with a swift, decisive blow. More like the bites of 10,000 mosquitoes coming 1 at a time, about every 5 minutes so you don't notice that they've stolen a significant amount of your blood until it's too late.

No, this is how creativity is shut out after one too many incidents where she wandered in, decided to be helpful and, due to imperfect understanding how things work, caused massive loss of productivity, money and reputations.

No, creativity is a process to be leveraged and managed like any other. This story is an account of life at a company where the process simply does not support the needs of the business. There are no excuses.

Excess code review rules and company policies are, in effect, a defense against the stupid. They're in place to limit or slow down the damage potentially done by stupid employees.

However, they're also a defense against the smart.

If there's a barrier erected to weed out stupid changes the barrier works to weed out smart changes as well. The barrier can't distinguish smart changes from stupid changes, and the people operating the barrier can't easily do that either since their asses are on the line in case they miss a bad change. As you can see, even the word of a higher manager can't change that since the defenses are spread out so wide: there's always some additional policy that requires more and more authorization from more and more people.

Good systems always support skunkworks approaches where good programmers have an unofficial half-sanctioned way to subvert the policies when needed.

Ironically, work to prevent layoffs neatly identified all the people who should be laid off immediately.

This shit happens at my job and leaves me feeling depressed and at times questioning my own skills because I know and my bosses know that on my part, I just had to make a tiny, tiny change. But when you have a dozen such experiences, I feel like something is wrong with me because when the dust settles, this was a one line code change that took me a week to push to production. It also makes my bosses question my estimates. Of course, when I goto the other extreme and give a ridiculous estimate, that also pisses them off and makes them feel like I don't really know what I am talking about. Saying "I cannot provide an estimate" also does not cut it for the boss.

In the longterm, I can feel this killing a part of me. I have always been a start-up guy working for myself. Since getting my first full-time job, stuff that I would expect the start-up me to take an hour to do ends up taking days and weeks, even. I am aware of the (mostly legitimate) excuses to not take too much self-blame but at the end, it still feels shitty because I know, the start-up me knows, this should not have been a one week project.

Ah the perils of having a non-tech boss and coworkers and being the only tech guy. Reading stories like I experience at work is comforting in an odd way.

Here's how my typical story goes...

Boss: So can we make this little analytics thing your #1 priority for tomorrow?

Me: Yeah, as long as we don't have anything else for projectX(my #1 priority)

Boss: Well, do you?

Me: Not right now. But it can pop up any minute Jenny decides she needs something done.

--NEXT DAY--

Jenny: Hey this isn't working in ProjectX. Could you look into it?

Me: Sure. there goes an hour

Boss: I know you're busy but this is very urgent. Can we grab you for few mins for this meeting?

Me: Sure goes another 30 mins

...

Jenny: That thing you fixed. Can we undo it and push to production because I realized blabla?

Me: thinking wtf? Ok sure.

...

Amanda: Hey can you check if these 2 users signed up from same IP?

Me: You can check that in admin

Amanda: Where?

Me: The admin panel you emailed me info to when I joined the company..

Amanda: Oh right. Can you fwd me the email I sent, I can't find it. My gmail's not working right.

Me: Ok.

...

Before I know it, I have spent every day of my job troubleshooting shit to never really get in the flow of doing my own project assigned by my boss.

Boss(after a month): How's that project coming? I know you said it's a one day thing.

Me: Yeah but the next day we changed the specs in the meeting. Also, projectX is getting a huge rehaul.

Boss: But still...you said it's a week after that

Me: Yeah assuming I didn't have work for projectX

Boss: But you'll always have work for projectX

Me: Yeah that's exactly what I told you too

Boss: So where does this leave us? Can you finish it in next week? I don't think there is much work for projectX

Me: But...

Boss: You're the startup guy. We want you to push us to release stuff quick. When you give us deadlines and are so off, it makes us lose faith.

Leave work annoyed, put in a quiet weekend with no coworkers around and wrap up said project

This doesn't sound like a process problem at all, actually. Really it's the reverse: your work environment is a mess, no one has the ability to do their own job except to bug you to do things they need.

And that's a disaster too, but it would be helped by adding process, not hurt.

Also, dude: learn to say no. Learn to prioritize for yourself instead of expecting your slag pile of a management structure to do it for you. i.e. ask: "What things must I get done to make sure I'm successful long term?" (which is entirely different from "What is the highest priority thing I have been asked to do?") and make sure you prioritize your work to get those things done (which is an entirely different thing from "work on those things"). And if your workplace environment really doesn't permit that, move on.

And if your workplace environment really doesn't permit that, move on.

Doing just that :)

It's basically a startup with 10 of us where I am the only in house developer.

I've tried many things like asking folks to Skype me instead of walking over to my desk or calling my name. I've gone to great lengths to get em to understand why skyping to get my attention is different than other ways. None of these guidelines really lasted more than ana few hours.

On this subject: Whenever someone chats me in the following manner, I want to kill them. Straight up. Just fucking take an ice pick and jab it right through their skull:

    Other Dude: Hi.
    Me: Hello

A minute elapses

    Other Dude: I'm having a problem
    Me: How can I help

A minute elapses

    Other Dude: ACTUAL PROBLEM THAT THEY NEEDED HELP WITH
    Me: Oh, here is your resolution which took all of 15 seconds to give to you.

WHAT THE FUCK MAN?! WHAT THE FLYING FUCKITY FUCK?! YOU JUST BROKE MY FLOW AND STOLE 2 GOD DAMNED MINUTES! WHAT IS WRONG WITH YOU?! IT'S FUCKING IM, YOU'RE NOT GOD DAMNED STANDING NEXT TO ME! AND WHY THE HELL DOES IT TAKE YOU A MINUTE TO TYPE EACH THING ANYWAY?! DON'T YOU KNOW HOW TO TOUCH TYPE?!

I mean, CHRIST, if this happened once in a blue moon, fine, whatever, but it seems like EVERY motherfucker uses IM the same way. EAT DICKS.

...

I might have went a little overboard on this.

> On this subject: Whenever someone chats me in the following manner, I want to kill them.

Then refuse this, set your status to "don't bug me, if you have a question just bloody ask it" and check your IM when you've got a lull[0]. If it's not a question, kill the window it's not relevant. I don't need "social interaction" on IM when I'm coding, if the other guy can't get to the point screw him, I'm not interested.

[0] If IM is specifically a way to get you immediately, start blacklisting people who can't get to the point and waste your time and have they go through email.

>If IM is specifically a way to get you immediately, start blacklisting people who can't get to the point and waste your time and have they go through email.

But of course they won't use email, they need the answer NOW. No, they'll come over to your desk and start tugging on your shirtsleeve.

I much prefer for people to visit, and to visit people. But it requires being able to take 'no' for an answer, and being able to say 'no'. Or at least, "I'm in the middle of something else; I can help you in 10 minutes. Otherwise, talk to Joe."

This is why I stopped using all chat clients.

That is why I insist on communication via chat clients when I'm busy. I'm not immediately interrupted as I would be if someone walks up or shouts out as I can ignore the little flash of the chat window on the task tray for a few minutes, sometimes tens of minutes, while I bring my train of thought to a more orderly stop.

If people start with "Hi, I have a problem" and don't state what the problem is I just respond with "drop me the details and I'll have a quick think" and get back to my work. Even if they respond immediately to that it might take me ten or fifteen minutes to get look at their message again - that is their time wasted, not mine.

Some problems require a more urgent response so I'll accept a walk-up or a shout for those, and some problems are best illustrated by showing them than describing them, I just have to trust people to know what they are and what is less urgent. And of course a message will come in at a time when I am eminently interruptible, in which case people get an immediate response anyway.

I find it works except for people who insist on not following the process, but they soon learn I'm far more helpful if they don't irritate me!

> If people start with "Hi, I have a problem" and don't state what the problem is I just respond with "drop me the details and I'll have a quick think" and get back to my work.

If people start with "Hi, I have a problem" and don't state what the problem is I close the chat window.

My work IM's status clearly states "if you have a question, ask it, don't ask if you can ask", time wasters simply get ignored.

> Some problems require a more urgent response so I'll accept a walk-up or a shout for those

Absolutely.

> If people start with "Hi, I have a problem" and don't state what the problem is I close the chat window.

In the past I've found that leads to people reporting "I asked him several times over a few hours and got no response" which, if the person being complained to does not know the situation, could look bad for me.

If I feel like being more terse, I just give a one-word response: "Details?".

> In the past I've found that leads to people reporting "I asked him several times over a few hours and got no response" which, if the person being complained to does not know the situation, could look bad for me.

As I noted, my conditions are clearly spelled out as my IM status text, if they can't bother following simple instructions I can't bother with them.

I'd answer that "I have a problem" with "I have a solution".

I do this all the time and add, "I like this game. Your turn." It normally gets a smile (at least in person), but doesn't really help with the flow problem.

I had clients calling me and don't hang up until i get the problem fixed...

YOU JUST BROKE MY FLOW AND STOLE 2 GOD DAMNED MINUTES!

Actually, it's worse than that. The (average) time it takes to get back into flow is 15 minutes[1], so add the time to respond to them to that to get your wasted time. And if you get another interruption before the fifteen minute descent into flow is complete? Start over - another fifteen minutes. If this happens constantly, you will effectively never get any work done. There's more than one way to kill productivity.

[1] - See "Peopleware", Chapter 10.

I'm really bad on IRC with this kind of thing. Need to get to the point faster. Thanks for pointing that out.

Not "faster", immediately. Say absolutely nothing until you've typed out your question.

Would you send an email that says "Hi", then another that says "I have a question", then another with your question?

Would you burst into someone's office and start telling them all the problems you expect them to fix without even saying hi?

Have you heard of these things called sentences? You don't need to hit return between each one.

"Hi Dude, I've got this thing I need help with. Here is the description of what it is and what I am having a problem with."

It's not an issue if they put it in the same chat message, it's the "Hi", send message, "I have a problem", send message, "This is what the problem is", send message that drives me up a wall.

And if you REALLY need him to get back to let you know if he can help out immediately, you can send a second message along the lines of, "And sorry to be a bother, but I need help pronto. Please let me know if you can take a look at this right now." At that point, it's not a big deal if there's another message, because you haven't wasted time in relaying it.

At that point it's not really a chat, is it? It's an email.

This is what I'm guarding against. I haven't been a programmer long, but it seems inhuman to not try and be polite or have a tiny lead up to the question. Ironically, it's the exact same thing that makes me impolite.

First of all, you're not bursting into someone's office. Second of all, if you burst into my office while I'm working and start making smalltalk, I will, depending on my mood and whether or not I've previously judged you to be a useless person, either immediately ask you to get to the point, or throw you out.

Stop wasting my time. If sending "Hi" makes you feel better, fine, but the substance had better be on my screen by the time I look at the window, or you've wasted my time.

How do I know you're even at your desk? After I type in my problem how long should I wait for a response before giving up and doing something else?

The same way you know he's not at his desk when you type that useless "Hi": you don't get a reply. How long do you wait after typing a useless "Hi"? Wait that same amount of time after typing your relevant question.

Hm, I've just started to think, also for my personal use, that maybe it wouldn't be a bad idea to try scripting some automatic 'Hello, how can I help you?' response to /^[^\w](hi|hello)[^\w]($MY_NAME)?[^\w]*$/i :)

I applaud this post.

Tim Schafer, of http://www.kickstarter.com/projects/doublefine/double-fine-a... fame, has a note on his door until 11am in the morning saying, don't knock, writing. Could you make progress on projectY if you made a deal to not work on projectX until 11am? To not even open your office door?

(FWIW, this is a classic Quality of Service (QoS) algorithm, where traffic on ProjectY is given dedicated bandwidth (2 hours in the day) so that it isn't swamped by traffic from ProjectX.)

Door? Who has those anymore? In the interest of collaboration open-plan offices have become the norm, to the detriment of productivity everywhere.

Funny enough I didn't get a private office but we got walls built for a private office when the sales guy joined because he was disturbing everyone. If my non developer coworkers realized how disturbing it is to be in flow writing code to have someone tap your shoulder and start discussing a totally unrelated issue to what you've been coding.

My place of employment is similar to yours, I've moved myself up the ladder in order to start isolating and enabling my dev team the peace and quiet they need to be most productive.

What if, while you code, you think out loud, like really out loud... louder than you would talk in conversation? Perhaps if YOU bothered EVERYONE then maybe they'd put you in a private office? It's worth a try while you are looking for other opportunities...

i.e. "LET ME THINK HERE... IF VARIABLE V EQUALS ENUM ACTIVE THEN CALL DOHICKY ELSE DO NOTHING... WAIT IS THAT RIGHT?! NO NO NO LET ME RETHINK THIS?! I BET IF I REFACTOR THIS CODE I CAN..." All day long...if they tell you to be quiet tell them to "shut up" you're trying to think and they are ruining your thought processes and they risk causing you to slow down and/or enter errors and bugs into the code... "what do you mean me talking and noise in general is disturbing to you and your work? How they hell do you think I feel?!?!?" :)

That kind of passive aggressiveness will make your coworkers hate you and could get you fired because they know you don't have to do that.

Apparently that passive aggressiveness already exists in his work place thanks to his bosses and co-workers... but some how you all fault him and say he can't share that same love?

I was thinking more about other situations. I thought the salesman was isolated to an office because he has to talk on the phone, constantly, to customers and it was interrupting the developers. I've heard similar situations with startups and their CEOs. If I did that in my open space office, my coworkers would just get irritated and eventually I'll be talked about it by my manager.

In fact, in my current work environment, when we moved offices there was a "central" open area with a view half built cube walls. Originally our telephone support were going to sit there and the Dev team was going to get placed in an enclosed shared Dev "war room". Support squeaked and the timid dev team got stuck out in the open.

Supposedly since Support is on the phone, they would interrupt everyone (everyone else int he vicinity is sales and they each have private offices) since they talk aloud on the phone. Why's this bad? Because breaking deep thought vs. some noise in the background (which still also exists because of the leakage they have on the phone in a single enclosed room) is much much more unproductive. Worse, the Sales team often forgets to shut their doors so sound leaks out into the open area or between sales calls they congregate a few at a time and have loud conversations disrupting the dev team. Worst of all is if they are in their offices, quietly doing paperwork with their doors open then decide to yell to talk to another salesperson in another office.

I often get up and shut Sales office doors, and constantly have to hush people up. My timid dev team doesn't because, well they are timid and don't want to rock the boat and add friction to inter office relationships. (I can hear, I agree. I need to encourage my dev team to speak up more, perhaps I'm sheltering them too much.)

unfortunately in environments like the one the OP mentions, the squeaky wheel is usually the one that gets the grease. So, while attached to the trailer (company) you're best to squeak loudly while you look for a better vehicle to get onboard with.

But if you are planning to leave, what harm will it really do? When you have pocket aces, bet and play them!

EDIT: Maybe you'll inadvertently help the next guy. Yeah they might not care much for you but maybe, just maybe, something will click and the folks will realize... yeah I guess disturbing him was stupid on our part. If not and they don't get it... who gives a crap about people that don't give a crap about you!?

That's not playing a good hand, that's being a dick simply because you can probably get away with it. Don't be a dick.

I sometimes am forced to do this at work. Just to get people to shut up from chit-chatting/small-talking very loudly. But then, I work in a cost-cutting, off-shored, cubicle farm.

Often, when we are pairing at work, the person driving is thinking out loud or the pair are in discussion with 8 other pairs in their row of tables.

If you only you could talk to them?

Seriously, most of these problems are due to lack of communication, set up a ticketing system, teach people how to use it.

Be a grumpy bastard. Someone taps me on the shoulder, I'm grumpy the first time, I growl the second time and if there is a third time I'll shout at them. Hasn't come to the shouting yet (although one guy got pretty close, but I just glared at him last time he approached and he grasped it).

Also, ask if you can work from home since your office environment is bullshit for writing code.

Work from home was turned down pretty quickly though I'm not sure if that would have addressed the root problem. I think the root issue is they need a developer who just does tech support for the other 10 non tech people. And then a developer who can actually build shit quietly.

If my whole job was to do tech support all day there would be no problem. Of course it wouldn't be fun and I'd prolly end up leaving, the present situation hasn't been much better and I'm on my way out.

Now that I am leaving soon my boss is very interested for advice on how to attract tech talent. A part of me just wants to send him a link to this thread :)

"A part of me just wants to send him a link to this thread :)"

That's probably the single best thing you could do for him, if he is serious about wanting to attract (and retain!) good technology people.

Re: working from home

Been there, done that. The only difference is that they call me on the phone instead of coming up to my desk.

I'll frequently work from home in the morning or evening if I want to get stuff done; no coworkers there. Or if I need to concentrate, I'll leave my office and go find a massage chair or cozy nook and curl up with my laptop, or even go to a different building and work alongside a team where nobody knows who I am.

Only thing I miss is my 3 24" monitors.

So how well does Linux support that setup? Or you're using OSX at Google?

Linux supports 2 with no problem. I'm using a second computer and synergy for the 3rd.

I think there's a definite benefit to team offices, but the team needs to be focused on the same project.

Wear some sort of ridiculous Productivity Hat with a door on it somewhere and an 'Occupied' sign.

the key part of that story is that Tim Schafer has a door.

Did you respond with anything other than "I asked you to skype me" when they ignored your request? If not, there's your problem.

People do what works. Reinforce their bad behavior, and you have a problem.

Yeah but I lost that war because my boss hates Skype and doesn't get why someone sitting inches from me should have to Skype me.

"It breaks my concentration. That makes things take more time. That makes things cost you more money."

edit: Suggest that he ask you if you're busy over skype. Talking face to face is more effective, and I'd hate to try to discuss something over it, but interrupting flow really sucks.

Because bosses and managers are suppose to support their direct reports by obtaining a work environment and tools that best enable them to be productive... if he refuses to contact you via skype, when he asks you a question verbally, respond by typing a skype message. It's not okay for managers to treat others how they (the manager) wishes or wants to be treated... great managers treat employees the way the employees wish to be treated (as long as that treatment is ethical and productive).

It can piss people off and so work negatively (so, evaluate your individual circumstances), but you can begin insisting upon written requests. Amongst other things, this documents the requests. (And you can tell people the written request is e.g. for your email-based "todo" list.)

If/when people start getting into your face (perhaps even formally) about performance, you have documentation to back up your argument. (Although, ultimately, most arguments are kind of moot, unless they end up in court or perhaps before an unemployment insurance evaluation -- not something you really want, anyway).

As for the overall experience. While negative, emotionally, you can still view it as a learning experience. Keep in mind that when you are in a better position, with more control, 90 % of the world with which you are competing is / will be what you are now experiencing. So... if you can cut out that bullsh-t, you will have a very significant competitive advantage.

It can be useful, to get a good, hard, close-up look at what you don't want. Just don't let it go on too long.

P.S. A trick I'll occasionally use, in problematic verbal situations, is to summarize the conversation/request in an email to the other party(s). 'We met. Here's what you asked me to do / how I understand it, what I'm going to do, and my anticipated schedule and potential conflicts / other overriding priorities. Let me know if I've misunderstood or you disagree.'

If no one responds, I proceed accordingly. If they bitch, hey, I laid out my approach and circumstances up front and asked for feedback.

It can put people on the defensive and create a negative vibe. But hey, when people tell you / manipulate you with statements of "We're losing confidence", things are not too positive, to start with. (P.S. That's already the sign/signal to bail. "Bail, bail, bail!")

Sounds like you need a project/product manager type shielding you from BS. Get in touch if you're looking for new opportunities in Socal area.

> Learn to prioritize for yourself instead of expecting your slag pile of a management structure to do it for you

What you are suggesting is easier said than done. I think that most good developers have the mentality you are describing...at first. When you work in an environment like this, however, you get worn down. You can only put up a fight for so long before you start getting all sorts of mixed up and you don't know what the f to do. If you took the best prioritizer in the world and put them in a corrosive environment like this one, the environment would eventually win out. I really feel for guys like the one that posted this original article and some of the subsequent commenters. I have been in that boat before and it is not fun.

Just out of curiosity, and this is something I've been meaning to ask for a while, how does the whole startup only having one tech guy (who isn't a founder) work? I've seen this scenario referenced many times on HN and I'm still having trouble picturing what the incentives are to work for such a company (as opposed to being made a cofounder) when it sounds to me like you're doing at least as much work as they are. I would normally think the sum total of all development (I'm lumping things like specs, coding, documenting, and testing all together) to be at the very least close to 50% of a startup's value, so if there is only one developer, he or she would seem to be worth about half the company. Is this kind of startup less software oriented than the kind I'm describing?

This will keep happening if you keep saying OK.

Try something like, "Can I get back to you in an hour? Boss wants me to finish this top priority item for him."

And if Boss asks you if you can finish it next week, say no. You're being asked to give an estimate, and it doesn't help to say "Yes, if [this thing doesn't happen which I know it will]". If they want you to be the startup guy, then act like you're the one in charge of your own schedule.

That works until they goto the boss and the boss comes to me...

"So I know your project is important but projectX is still our #1 priority so to the extent Jenny needs you please prioritize her work"

A better trick I discovered when I used to work in an environment closer to this was to always give the person requesting help some trivial but sincerely related task to do before you can help them, e.g. "can you just write down all the steps in the process before you get stuck, it will help me solve the problem", or "Hmm... I'm not quit getting the issue, can you draw me a diagram and then get back to me". This acts as a great filter for people that are just helpless versus people that really need help. And if anyone complains to the boss that you aren't helping, you can say "oh well did you do task X? that's really all I need from you to get started"

One day I had an epiphany. I realized as I was programming at my computer that nobody had a freaking clue what I was actually working on. If someone asks how their low-priority project is going, I just say that I'm still working on it. They have no way of knowing if I'm actually working on it right now. In the real world, saying "no" a lot will get you enemies. It's much more diplomatic to only work on some low-priority project an hour or two a day (you don't have to tell them your are only spending an hour or two a day), or say you'll get to their project immediately after you finish this thing your are working on (which can be a small component of your large important project). It's important to know how to say "no", but in practice it is important to minimize the number of times you say "no".

Then you either demand dedicated "anti-office hours" or you tell them that it's not feasible that project B will get done with the way things have been recently. They're not technical, and all they can do is take your word on when you think you can finish it. They're not going to pick up on the caveats you've laced in your estimate. Also, you should let them know ahead of time that things are slipping instead of them finding it "behind schedule" on their own.

Honestly, don't take this as an insult, but it just doesn't sound like you're the right person for the position. If you have trouble pushing back when you need to, then you should find a spot where there is someone pushing back for you.

I agree with you that I'm not the right person. I was brought in to change the culture so we do everything faster(you can imagine the speed at a company with couple offshore devs and half dozen onshore people working on product).

I've, err we, have mostly failed.I can go on about why but at the end, I think we all share the blame and evaluate parts we could have personally done better and apply it next time round.

One of the big realizations for me is that they need someone older and more confident. I'm in my mid 20s and once founded a startup that got 30x the traffic but even months in, the boss would be unsure if I was a good programmer. We tried everything from emailing my code samples to his CTO friends to code reviews by other devs(all of which gave positive feedback) but I feel he could never totally shake that doubt. I called him out on it finally(basically if you can't tell after 8 months if I can program you should let me go). I've been programming since 12 but the lingering doubts actually started making me question my own abilities at which point I decided I need to move.

I found keeping a timesheet helped me there. When it came to "it is getting to the point where X is not going to be ready on time, and I've got personal things on this week so I really can't work significant overtime", and the question came back "well, what have you been doing for the last week", I could give them a full list of "2 hours helping X with Y", "an hour doing Z", "four hours on that high priority fix client C that no one else was willing to touch" and so forth. Don't play the blame game and point out that person X was being thick on an occasion or otherwise dwell on who is interrupting your key work, just lay it out as matter-of-fact as you can: "This is where my time is being used".

Make damn sure though that people understand a short interruption "for a quick meeting" when you are at the time "powering through" stuff sometimes kills as much as a full hour of development time if you are working on something complex as it can completely break your train of thought (especially if you are tired due to recent overtime!), or worse if you are coordinating with other people and/or are working on something timing sensitive (so the interruption might mean a key coordination point is delayed).

If you can show what the demands on your time are, a good manager will try rearrange things so that you are interrupted less often. Sometimes simply merging those small interruptions into a scheduled period or two each day is enough to make all the difference to your mid/long term work: problems that didn't really need you magically go away (as the person who was going to interrupt you does that 30 seconds of extra thinking/research that was needed to give them the clue) and the others "interrupt" you a few times a day rather than several times over an hours or two.

In which case you then ask them to send it in an email, and CC your boss so there's documentation.

Let the managers fight it out.

Here's my suggestion: have your boss talk to Jenn and make her understand that you'll be spending the next day working on the boss' request. Any interruptions would have to go through the Boss, so that all parties are aligned.

This seems like a communication problem between the managers. (I can speak from experience, since I messed this up once when I reallocated our engineers' time without fully being on the same page as the engineering manager)

Ideally that should work but projectX is what the whole company is working on and key pieces of it are being tested throughout the day so it's impractical to have a cut off time.

Well the converse of my suggestion adapted to your particular situation would be this:

Boss: Could you do this thing for me?

Badclient: Well, as you know, I have Project X as my highest priority. I'm happy to oblige, but Jenn has dibs on my time at the moment. If you can talk to her and come to an agreement on how I should distribute my time, I'll be happy to work on your thing.

Whatever happens, your boss is on the same page as Jenn, and all you have to do is execute the agreed upon plan.

1. Never tell anyone how long things take until you got a chance to sit down and make an estimate vs. guesstimate.

2. Never let anyone get you in the position to set priorities yourself (unless you're the boss). Refer them to your manager or whoever is highest in the chain of command and assigned you a task to make this decision.

This is really more a working culture problem. You'd think it's in the management's best interested to make sure nobody ever violates 1. + 2.

One backlog for new requirements, another backlog for incidents. Track your time working on the different backlogs (or even per-tasks) and pretty soon you will have some numbers to show the boss why new stuff isn't happening snap-snap.

ps: Now that I am leaving soon my boss is very interested for advice on how to attract tech talent. A part of me just wants to send him a link to this thread :)

I mainly stuck around because it was my first FT job and I'd promised myself I won't bail on it for one year no matter how bad. In past I've regretted quitting on projects too quickly...at least I won't have that regret here.

For a job like that? Pay them. A lot (a good chunk of the company might suffice). Or suffer an extremely high turnover again and again and again.

At my previous company we had the same problem, where people from all over the company would come up to us developers constantly asking us to do things which were great, but not really what we were supposed to be doing. Like you, we never got around to doing something that we said would only take a day.

One week we all decided to keep a notepad and paper next to our desks and keep a log of every time we were interrupted and asked to context switch and do something random. It was so bad, before the end of Monday our boss was so dumbfounded he completely re-jigged everything so it wouldn't happen like that again.

We ended up calling one developer pair the "consulting developers" each week, and only that pair could be interrupted. There was a huge sign above their desk advertising them for the week. We sacrificed one for the good of all, and it worked really, really well. AFAIK, they are still doing that.

This is great to hear. It is pretty much the problem/solution that I've diagnosed and stated in another post:

I think the root issue is they need a developer who just does tech support for the other 10 non tech people. And then a developer who can actually build shit quietly.

I will push this stance more aggressively having read your post.

As much as this sounds counterintuitive, say no, and stop being a team player.

Petty requests for emails that someone should have are their responsibility. If you think about it, why should your resourcefulness help them complete their job while it blocks your own progress?

Your boss is doling out reviews, he's #1 priority. Everyone else gets "no" and no access to your time without talking to him first.

"Being a team player" is a fluffy way of saying "do other peoples jobs for them". Sure, help people out and don't be a dick.

It does sound like a mess, but this type of mess exists everywhere. When you meld highly technical people with less technical people you will get these scenarios. Before you know it, you will be expected to replace the toner.

One of the main reasons I took a FT job was to become a better team player so in the long scheme of things, I always wanted to make this work out. Obviously it hasn't worked out too well but I think I have learned some critical lessons for my next gig!

Yak Shaving as an institutional affliction: http://projects.csail.mit.edu/gsb/old-archive/gsb-archive/gs...

Interesting; I've encountered "yak shaving" as meaning something slightly different: it's when your main task depends on a subordinate task which is long, difficult, and stupid--e.g., in programming, it's something which the tools should be doing for you. (Bonus points if you know that, on a different OS, or in a different language, the tools would be doing it for you.)

The original MIT meaning is closer to what I recently heard described as meta-yak-shaving, which is when, before you can start shaving the yak, you discover that each hair of the yak contains a subyak, and each subyak has to be shaved individually.

This is something the Scrum methodology is supposed to solve. It places the responsibility of prioritizing tasks at the hands of the stakeholders by making the tradeoffs between scheduled tasks and interruptions very explicit.

If you want something more light weight look into the Pomodoro technique. People can bring issues to you but only in breaks between blocks of uninterupted work.

If you go with either then you can explain away refusing to be distracted as merely following a proven methodology. This gives you an easy out as people will vent any frustration at it rather than you.

Your problem is solved by a single word. Copy & paste the following : No. Alternative: No, sorry.

You don't look bad for declining requests. You will get your shit done. Just learn to say no. Really.

Just curious how many other folks see similar patterns of interruption? (i do)

I see the familiar pattern of programmers not saying "no" and not explaining clearly what they need to work.

None of the other people here are being particularly unreasonable, they're just ignorant. And given that "how to work with programmers, writers and mathematicians" is not part of any educations, it's not really their fault.

Nor is "how not to be interrupted" covered in the CS curriculum.

My gmail's not working right. -- Hahahaha. I hear this all the time.

Solution: stop taking requests from people. Tell them to file tickets, and your boss will have to prioritize them.

No, don't quit! Embrace! That's 14 hours (if I remember correctly) you could have spent working on your own projects instead of being pissed off on HN. The world is full of this, and it's well accepted and nobody questions it - make the most of it.

They wouldn't be his projects. They would be the company's.

I agree with you here. That's a ton of free time. Unfortunately Ed probably posts one disgruntled story each month. I predict he'll post one next month too. He wont quit, he'll just complain on Hacker News.

Good point. When I'm stuck and management takes a couple days to get back to me and I have nothing to do, I take advantage of this time to learn new things. Maybe I'll read the man pages for "find" and discover a few new tricks. Or I'll read some tutorials on vim and learn some new things. I've learned a lot about MongoDB this year while waiting on roadblocks at work.

I just went through the same thing. Except I was smart (or so I thought) and had what I needed changed in a config file.

PHB decided that it still counted as a "code" change because someone needed to log onto the box and change the config. Literally it was removing 3 characters from it. The only reason I even had it in the config was to avoid it being a code change to begin with. I argued back that in the time I would take to get through the whole process I could write a screen to do the same thing.

PHB said that would be fine as from that point on its a config not a code change. So off through the whole process, where I present the change to people who don't care, or understand the change and ask for them to approve it. I get that they need awareness, but seriously couldn't a short email saying X is going to happen at time Y suffice?

Im tempted to embed a Python/Ruby interpreter into my next project with a web interface to modify the code. Then I can do these minor changes without going through the 7 day process.

the difference between config and code is largely illusionary, so PHB is right: both can break things and need to be tested.

I would argue that in this case where it was designed like that from the beginning IE detailed as how the system works then there should be no issues.

I did forget to mention it did go though a round of testing, which took less then 5 minutes to be verified as correct. It literally was 6 days of process wrapped around 10 minutes work.

Why stay?

Mostly due to certain parts of the job being very enjoyable and that this sort of thing isn't that big a bugbear for me to look... yet.

'Fuck that shit.'

maintenance programmer words to live by.

Why is the author still working there?

It's exactly this situation that made me leave a large company after only 5 weeks. I'd come from a smaller, more agile environment and having to go through multiple layers of process and bullshit just to get the simplest thing done was just too much for me. I know 100% that if I'd have stayed there I would have ended up hating my job and affecting my passion for writing code.

The main situation I remember is a code review of a few lines I'd written kept on being rejected by some of the more process-indoctrinated programmers purely because I didn't use the correct type of verb in the first sentence describing my change! I mean, when that is the shit that matters and not solutions to problems, I'm out.

"It Takes 6 Days to Change A Fundamental Business Practice" sounds a lot more reasonable.

Should you be able to extend the backlog of a factory by an entire month just by changing a variable, pushing a build and going for lunch? No.

The problem, imo, is not that it took 6 days of reviews to change a major variable. The problem is that the reviews were entirely immaterial to the actual change taking place. Whether variables are hardcoded, have an audit trail, and use proper naming conventions will have no impact on ACTUALLY mitigating the business risk of making a giant change.

(rewriting this a bit to make it clearer).

It's clear that a mission critical change needs oversight and risk mitigation. But think of the waste of this process from the Lean perspective: Waiting, Handoffs, Checking, Inspecting, Obtaining approvals, Reviewing, Filing, and Rework.

IT groups need shepherds to be able to guide these things through the system of checks and balances. If you look at this scenario, Ed, the programmer, was the orchestrator. The end-to-end process was ad hoc, and not designed purposely. There were actually several processes at play, designed by different departments (IT demand, support, delivery, QA, etc.), with different value systems, and the interaction between them isn't well defined towards the end value of "shipping software changes that work". The division of labour across review, testing, etc., without incentivizing end customer results, and minimizing waste, has devolved into a bunch of school marms that redline all documents or code given to them without actually working to help with the end goal.

The "this test plan isn't good enough" for example is a near and dear to me. Why can't QA dedicate a resource for a brief period to help make it better? Or at the very least, give guidance on the what they want to see for approval? Usually this (and ornery change management boards) wind up being the primary source of senior management overrides - the QA group doesn't improve quality, it just slows down change thus maintaining the current (functional) mediocrity.

IF the process was shepherded by a manager to actively involve QA, Code Review, Change Management, etc. earlier, Ed might be less frustrated, and it would have been done in 3 days. ;)

This happens waaaaaaay more than anyone will admit.

Is it sad that I think 6 days is pretty fast?

I've worked places where 6 days would be a miracle. For me, the worst has always been when the database query plan suddenly switched to something with a full table scan (I'm looking at you Sybase) or the always fun full sort merge (nice going Ingres). It always seemed to come down to forcing indexes which tended to get deployed fairly quickly.

Yes, it's very sad. I get kind of twitchy and nervous if code that I have written doesn't get tested and into production same-day or next day.

So do the reasons why such infuriating processes are enacted.

Look on the bright side: if large companies weren't this screwed up, small businesses and startups wouldn't have a chance.

I wonder if the place is covered by something like ISO 9000?

The real WTF is that at the end, all it takes is the president giving a direct order to put the code in production immediately. That is, all the policies and processes just add noise to the interface between strategic decisions and operational tasks. That's how big companies fail.

This happens at my job too and I want to kill myself when it does. I get so frustrated. All these work protocols and slow useless software that prevents me from being effective at work.

I usually find a way to make things work by going outside the protocols and just fixing the problems. Its not the problem that is hard, its the stupid damn protocols and official ways of doing things that makes it very hard to get something done.

Decisions being made by idiots who have no clue how to allow people to work in a efficient manner, or how much money they lose by people sitting around doing nothing, waiting for some useless step in a useless process.

Seriously, I can feel the rage coming just thinking about it while writing this! What the fuck happened to this industry...

Heh, not the same corporate snafu, but my most expensive work was done to change 1 char.

Spent about 3 weeks with a new codebase to find that one number had to be increased. Funny thing is, there was a comment there and somebody had increased it before when they had a similar problem.

Time to change the character: 10 seconds

Time to find out what charater to change: 3 weeks

I've found that bugs that are very hard to debug often have the smallest fixes. Once you've discovered what tidbit of information you were missing, the fix is trivial. I guess this is a needle/haystack problem.

Better that than they push the line of code and everything stops working for a fortnight!

What I identified with the most is the requirements creep that happens in QA/validation

This is a case where programmers are overhead not production and the "system" is sized to handle peak loads.

Another way of looking at it, "It Takes 6 Days to Prevent Layoffs."

That's pretty damn efficient from a business standpoint.

And pretty damn important from a human one.

While I can both relate and empathize with the story here- the initial "crisis" seems forced and arbitrary. Once a company becomes sufficiently large to mandate a promotion process like the one described in this story, the management would probably know that there is no "immediate" changes that can be made unless they want to bypass those processes. Further, if it took 6 days to change a line of code to meet everyone's standards and satisfy requirements, imagine how long it would take to push through a layoff. :)

I understand the advice to quit because of the agonizing process involved, but worse than that for me is the idea that this parameter is hard coded, and even with this change in place that limitation still remains. NO WONDER the change process is so painful, the software design is clearly brain dead. I bet all kinds of unexpected shit happens when changes are rolled out. This is a code smell like a rotting corpse.

Moving parameters somewhere else doesn't change the problem at all, only moves it to a different file. Altering a parameter that effects a large chunk of the system still needs the exact same amount of testing to be done, all you save is a compile.

For what it's worth. I'd also say that anyone who thinks that hardcoded parameters are always bad, probably hasn't worked on anything large or complicated enough yet. There are plenty of cases where you've got parameters which could theoretically change, but the likelihood is small enough that it doesn't make much sense to move it out to somewhere else - not coincidently, these tend also to be parameters which effect large chunks of the system and should never be changed without a solid business reason and a full QA cycle.

As good a 'system' as you might have, it is still no match for having a middle manager with a clue (and some freedom to move within the system)...

Sure - this should have still been tested (and thoroughly!), but putting this 'on the queue' when it needs doing now? Making the programmer deal with other issues (rename the variable) while doing it? Not making sure that a testing machine was available? No....

I have to opposite problem, almost no process at all.

I'm not sure which is worse, having to wait 6 days to do a change or having people request (often trivial) changes to be deployed the same day they specified them and having people going round editing .php files directly on the production server.

THANK YOU.

This couldn't have come at a better time. Describes my current situation perfectly.

I changed 1 line of code today. It's going to require me to call into 2 review meetings to have it approved. These meetings happen once a week. 2 weeks to get this code to production. UNBELIEVABLE.

I think it's time for a new job.

I can relate to that. Maybe 70% of the defect corrections I make in our highly complex and optimized code base are less than 10 lines worth of code.

It can take between between 24 hours and a week to figure out what those few lines should be, though. And that is extremely frustrating.

> Pissed off hours spent on Hacker News: 14.

Seriously I've already mentioned this for your last post, but imo it's really time to leave your job for something else Ed. Unnecessary stress lessens the quality of your life, as well as lowering its total lifespan.

Ditto for most of the IT services companies in Bangalore. Oh just add "Project Manager X chasing the required parties" between every line in that article and perhaps add another 3-4 day delay as most of the teams don't work in the same time zone.

I thought the last line would read "Weeks of notice given: 2"

Why are you putting up with that crap Ed?

This is the model which my Computing coursework bases around, you get something like 6 marks for code and 8 marks total for a "test plan"

The best thing? I think I've lost the majority of marks for using RSpec instead of 100 screenshots

This happens at both big and small companies; I worked with a company of 30 people who had this 'in place' already because the CTO was had 'experience in leading very large projects'. It was kind of annoying.

I'm very lucky to work in a place where they understand the limitations of this policy.

Most of the product we work on (not a CRUD app) is finished or at least at a fairly mature state. But the area I work on is not. For a while they harped on us about following the process. But due to their lack of interest on what we were doing and some managers protecting us, we ended up able to make rapid changes in response to test failures, often several times a day.

It was incredible. They would seek investigations through change requests on bugs found in testing. Immediately after they were entered - before the change request even went to the overseeing group of project engineers and subject matter experts - we would tell them what caused the problem and write that we had committed a fix to the mainline. This circumvented days of waiting for a response that would have blown our schedule apart.

It paid off big. Today for the first time our tests all started passing. They've been failures for years and no one expected anything close.

There's a time for process. When a lot can break or you have a finished/safety-critical product you want to lock it down. But in initial development of new features, you have to throw it out and do what's right.

For a while they were demanding change control with CRs over software that had never been written or run successfully. Why even bother writing the CR if it never worked in the first place? That means the software wasn't done in the first place, not in a state requiring change control!

It was unusually rapid iteration between the test and development teams, who should sit next to each other and immerse themselves in making the software work. Only by rapidly uncovering and overturning problems like this can you ever get something close enough to stable to lock it down this way.

Development and test respected each other, understood what had to be done, and by working together discussing behavior and the test environment were able to determine what the software should do and how best to test this functionality.

There were times where waiting for the process would have taken us 5 days to get approval to fix a bug. By talking with our project engineer and going ahead with the testers - who realized it was not the time to stick to formal process - we were able to succeed.

I'm very lucky. The testers were cool and understood the state of the software and were willing to shout across the cube and explain what was failing. We worked together and managed to succeed by knowing when a process was important and when the managers would be happier to go around it.

The problems encountered here are mainly about people. We had testers and others who trusted and worked with us. When you don't have that - like the author of the article - it takes some social engineers and lawyer like knowledge and parsing of company policy to get around the problems.

I find it troubling that they had to make a change at the point of layoffs to create a solution to prevent it. Couldn't they have discovered this trend before it got to that point?

This is how individuals working with open source software beat a large corporation where by all accounts they should not stand a chance. Bureaucracy is the Achilles Heel.

Is this a story or a real thing that happened?

This is an unfortunate reality. People really do not care about all the work you have just the things they want done.

I smell opportunity. Companies that are as archaic and inflexible as this deserve to be outdone.

This reads eerily like my job, just s/code change/system change request

Love the my job, hate the bureaucracy.

"you are responsible for fixing preexisting errors that violate new company policy"

That's insane.

Seems like the "boy scout rule" got out of control.

Why aren't you appealing directly to Philip more often? Give him a list of what you need and he could have what he wants with 3 emails. Also I don't understand what "access" you need to Marge? You can't email her or something?

edit: d'oh

Homer and Marge are most likely some kind of testing systems.

I could be wrong, but I think Homer and Marge are servers, not people.

I thought it was pretty obvious that Marge is a computer.

"Also, dude: learn to say no." This is horrifyingly bad advice, and in fact I doubt it even acknowledges the problem. It takes time to say "no". Are you telling us to learn "the path of least resistance" in saying "no"? Is this legitimate advice?

At a previous development gig, I would have to learn how to say "no" to at least 5 different types of people. And quite honestly, and candidly, I think this advice is absolute bullshit. I genuinely believe it wholesale ignores all the junk-news that's bubbled up around Zuckerberg's attire at meetings, the joke about Ohanian's suit on Bloomberg TV, etc., the culture of corporatism. If you say "no" enough, you end up like Zuckerberg: An Emperor with no Suit. (Or to put it less abstrusely: "This guy won't play ball.") And if you say it with the force and cavalier quality I think you're thinking: you get fired.

This advice prompts me as if a manager, if it's taken without the gusto of the frustration we're all feeling. And I don't want to be a manager.

Let me put it this way. Are you saying there's a verbal silverbullet that shoots down CEOs, Team Leads, Content Authors, Co-Developers, Designers, IAs, Managers, Testers, Team Members, etc.?

I suppose many others have already said this: _easier said than done_.

This thread reminds me of two lessons that I'm thankful I've learned young:

1. Processes exist for a reason. They're often annoying and sometimes overkill and unnecessary, but they're often there as a result of a past problem.

2. The whole top thread about communicating and literally ignoring people reminds me of my roommate who bragged that he was a good leader, meanwhile telling me that he would simply ignore his teammates because they annoy him. This is why "geeks" get the stereotype that they can't interact with people and it's why programmers with business and communication skills are so valuable and sought after.

6 days you LUCKY LUCKY Bastard - i have user stories over 6 months old that haven't been actioned yet and had to wait 2 months to get a gwt verification file installed

is it only me or there is a (just one) line of code gone wrong on ed's comment handling server?

{"post_id":131434793,"html":"\u003Cdiv class='p_response_container' style='display:none;'\u003E\n\u003Cheader class='clearfix'\u003E\n\u003Ca href=\"http://edweissman.com/it-takes-6-days-to-change-1-line-of-co... class=\"p_view_all_link\"\u003EView All 16\u003C/a\u003E\n\u003Ch1\u003EMost Recent Responses\u003C/h1\u003E\n\u003C/header\u003E\n\u003Cdiv class='p_responses_list clearfix' data-posterous-responses-container\u003E\n\u003Carticle class='p_comment p_response' style='display:none;'\u003E\n\u003Cdiv class='p_info'\u003E\n\u003Cspan class='p_icon'\u003E\u003C/span\u003E\n\u003Ctime pubdate='1337295601'\u003E14 minutes ago\u003C/time\u003E\n\u003Ca href=\"http://tomakefast.com\\u003EPJ Brunet\u003C/a\u003E responded:\n\u003C/div\u003E\n\u003Cdiv class='p_comment_body'\u003E\n\u003Cdiv class='p_profile_photo'\u003E\n\u003Cimg alt=\"\" src=\"/images/profile/missing-user-35.png?1337287723\" /\u003E\n\u003C/div\u003E\n\u003Cdiv class='p_comment_text'\u003E\nHilarious, I love it.\n\u003C/div\u003E\n\u003C/div\u003E\n\u003Cul class='p_action_links'\u003E\n\u003C/ul\u003E\n\n\u003C/article\u003E\n\n\u003Carticle class='p_comment p_response' style='display:none;'\u003E\n\u003Cdiv class='p_info'\u003E\n\u003Cspan class='p_icon'\u003E\u003C/span\u003E\n\u003Ctime pubdate='1337296138'\u003E5 minutes ago\u003C/time\u003E\nkkudi responded:\n\u003C/div\u003E\n\u003Cdiv class='p_comment_body'\u003E\n\u003Cdiv class='p_profile_photo'\u003E\n\u003Cimg alt=\"\" src=\"/images/profile/missing-user-35.png?1337287723\" /\u003E\n\u003C/div\u003E\n\u003Cdiv class='p_comment_text'\u003E\nI agree and disagree at the same time. It does take a ridiculous amount of time to put something in production, unfortunately, but the scenario you described above is probably an exceptional case.\n\u003C/div\u003E\n\u003C/div\u003E\n\u003Cul class='p_action_links'\u003E\n\u003C/ul\u003E\n\n\u003C/article\u003E\n\n\u003Carticle class='p_comment p_response' style='display:none;'\u003E\n\u003Cdiv class='p_info'\u003E\n\u003Cspan class='p_icon'\u003E\u003C/span\u003E\n\u003Ctime pubdate='1337296154'\u003E5 minutes ago\u003C/time\u003E\nbob responded:\n\u003C/div\u003E\n\u003Cdiv class='p_comment_body'\u003E\n\u003Cdiv class='p_profile_photo'\u003E\n\u003Cimg alt=\"\" src=\"/images/profile/missing-user-35.png?1337287723\" /\u003E\n\u003C/div\u003E\n\u003Cdiv class='p_comment_text'\u003E\nDoesn't inventory cost $ and space? Maybe CFO should be involved too.\n\u003C/div\u003E\n\u003C/div\u003E\n\u003Cul class='p_action_links'\u003E\n\u003C/ul\u003E\n\n\u003C/article\u003E\n\n\u003C/div\u003E\n\u003Cdiv class='p_comment_form p_logged_in'\u003E\n\u003Ch1\u003ELeave a Comment\u003C/h1\u003E\n\u003Cform action=\"/responses/create\" class=\"new_comment clearfix\" id=\"new_comment\" method=\"post\"\u003E\u003Cdiv style=\"margin:0;padding:0;display:inline\"\u003E\u003Cinput name=\"authenticity_token\" type=\"hidden\" value=\"F6F7vP+9I0xtnokVneiKN6klXpPRM8ziSWl/D/arZxI=\" /\u003E\u003C/div\u003E\n\u003Cdiv class='p_not_authorized p_comment_section'\u003E\n\u003Cdiv class='p_additional_fields'\u003E\n\u003Clabel for=\"comment_name\"\u003EName:\u003C/label\u003E\n\u003Cinput id=\"comment_name\" maxlength=\"80\" name=\"comment[name]\" size=\"40\" type=\"text\" /\u003E\n\u003Clabel class=\"p_comment_email\" for=\"comment_comment_email\"\u003ELeave this field blank to comment.\u003C/label\u003E\n\u003Cinput class=\"p_comment_email\" id=\"comment_comment_email\" maxlength=\"80\" name=\"comment[comment_email]\" size=\"40\" type=\"text\" /\u003E\n\u003Clabel for=\"comment_toast\"\u003EEmail:\u003C/label\u003E\n\u003Cinput id=\"comment_toast\" maxlength=\"80\" name=\"comment[toast]\" size=\"40\" type=\"text\" /\u003E\n\u003Clabel for=\"comment_url\"\u003EHomepage:\u003C/label\u003E\n\u003Cinput id=\"comment_url\" maxlength=\"80\" name=\"comment[url]\" size=\"40\" type=\"text\" /\u003E\n\u003C/div\u003E\n\u003Caside class='p_login_options'\u003E\n\u003Cdiv class='p_asterisk'\u003E\u003C/div\u003E\n\u003Ch1\u003EWant to skip this stuff?\u003C/h1\u003E\n\u003Cp\u003ELogin with any of the following:\u003C/p\u003E\n\u003Cdiv class='p_login_buttons'\u003E\n\u003Ca href=\"http://posterous.com/login?flow=newcomment\u0026amp;jumpto=h... class=\"p_posterous_login\"\u003ERegister or login to Posterous\u003C/a\u003E\n\u003Ca href=\"#\" class=\"p_twitter_login\" data-posterous-jumpto-url=\"http://edweissman.com/it-takes-6-days-to-change-1-line-of-co... data-posterous-post-id=\"131434793\" data-posterous-redirect-url=\"http://posterous.com/oauth/init_oauth_and_redirect/?ssod=edw... data-posterous-twitter-login-button=\"true\"\u003E\u003Cimg alt=\"Sign in with Twitter\" src=\"/images/site/sign_in_with_twitter.png?1337287723\" /\u003E\u003C/a\u003E\n\u003C/div\u003E\n\n\u003C/aside\u003E\n\u003C/div\u003E\n\u003Cdiv class='p_comment_section'\u003E\n\u003Clabel for=\"comment_body\"\u003EComment:\u003C/label\u003E\n\u003Cdiv class='p_comment_area'\u003E\n\u003Ctextarea cols=\"40\" id=\"comment_body\" name=\"comment[body]\" onChange=\"if (this.scrollHeight \u0026gt; this.clientHeight \u0026amp;\u0026amp; !window.opera){this.rows += 1;}\" onKeyPress=\"if (this.scrollHeight \u0026gt; this.clientHeight \u0026amp;\u0026amp; !window.opera){this.rows += 1;}\" rows=\"5\"\u003E\u003C/textarea\u003E\n\u003C/div\u003E\n\u003C/div\u003E\n\u003Cdiv class='p_comment_section'\u003E\n\u003Cinput id=\"comment_post_id\" name=\"comment[post_id]\" type=\"hidden\" value=\"131434793\" /\u003E\n\u003Cdiv class='p_submit'\u003E\u003Cbutton button=\"true\" class=\" action_button\"\u003E\u003Cspan class=\"\"\u003EPost this Comment\u003C/span\u003E\u003C/button\u003E\u003C/div\u003E\n\u003C/div\u003E\n\u003C/form\u003E\n\u003C/div\u003E\n\n\u003C/div\u003E\n","responses":[{"name":"PJ Brunet","body_html":"Hilarious, I love it.","created_at":"2012/05/17 16:00:01 -0700","body":"Hilarious, I love it.","post_id":131434793,"id":11028186,"display_name":"PJ Brunet","comment_type":"comment","display_url":"http://tomakefast.com,allowed:true,display_photo:/images/pro...},{"name":"kkudi","body_html":"I agree and disagree at the same time. It does take a ridiculous amount of time to put something in production, unfortunately, but the scenario you described above is probably an exceptional case.","created_at":"2012/05/17 16:08:58 -0700","body":"I agree and disagree at the same time. It does take a ridiculous amount of time to put something in production, unfortunately, but the scenario you described above is probably an exceptional case.","post_id":131434793,"id":11028197,"display_name":"kkudi","comment_type":"comment","display_url":null,"allowed":true,"display_photo":"/images/profile/missing-user-35.png"},{"name":"bob","body_html":"Doesn't inventory cost $ and space? Maybe CFO should be involved too.","created_at":"2012/05/17 16:09:14 -0700","body":"Doesn't inventory cost $ and space? Maybe CFO should be involved too.","post_id":131434793,"id":11028199,"display_name":"bob","comment_type":"comment","display_url":null,"allowed":true,"display_photo":"/images/profile/missing-user-35.png"}]}

Hey, thanks for wrecking page display for me! :)

it was just one line! :-)

What year did this occur? Do you get a lot of paper cuts from feeding the punch card reader?

Fuck. That. Shit. Seriously, I'm not exactly thrilled in my current startup role but stories like this give me a new appreciation for it.

Reply on news.ycombinator.com