How terrible code gets written by sane people

Just hire devs who are great coders, great communicators, stubborn enough to push back against upper management, good hearted enough to sacrificed their own KPIs to focus on the success of their project, and massochistic enough to stick it out at an obviously poorly run project.

Good luck

Side note, for 15 years of experience the project doesn't seem that bad by a long shot. I only have ~7 years of experience and have seen some far worse code. I was just recently on a project that has some methods that take 8 call backs(project was greenfielded in Jan 2016) before that I was on a project where the development team copy and paste the source for the last web page every time they needed a new one, but not delete the initialization code that fetched the data from.the database. So the 18th screen they worked on had 26,000 lines of code and took 5 minutes to load. And every button click triggered a reload. So, add a new widget and boom, wait 5 minutes for a screen refresh.

I've worked for long periods of time in 3 codebases that were over 2 million loc in my career, all had great conditions, all had very smart people. Those are not huge codebases by any standard, and yet all 3 had people talking about ground up rewrites. All 3 had people complaining about every single example the author used. Two of them undertook the ground up rewrite estimating one year. Both admitted to having made a huge mistake 3-5 years later.

My theory is that all software eventually becomes difficult to maintain and full of warts, regardless of smartness, regardless of conditions. We all have code ideals we believe and talk about that don't work as well in the real world as we imagined. We all have deadlines that are shorter than we want -- it's a universal constant. We all imagine we can write code and fix big problems faster than we really can. And we all fail to fully understand what's working right in a messy large codebase and tend to focus on what's wrong.

I don't know how to fix this, but it's very possible the author ran into thoughtful paradigms he just hadn't seen before and didn't fully understand. Some styles seem messy if you haven't seen them before.

React is an intentional lack of separation of concerns between code and markup. Some projects are required to mix tabs and spaces depending on tools. For that matter, bash scripts alone require mixing tabs and spaces for some features. And so on, it's important to have context for why things are the way they are before jumping to the conclusion that it's bad.

I once worked at a company that used an offshore company to work on certain modules of a large project. They would commit code that my team would have to then code review. We'd see things like large IF-ELSE blocks with minor differences among the conditions (sometimes just one char). I know everyone hates this.

Turns out their internal metric was lines of code per day, so they'd bloat the shit out of everything.

Second comes qualifying for the money (yes, in real world you get money without having what you are selling, but that's a short-term success). That means getting to ask for money again tomorrow. That is where skill and code come into play. You promised something and now when you are lucky you can deliver 80% of that. Push hard to get there. Not everybody who tries will achieve that much.

And only then comes quality, which means qualifying for money next year becomes cheaper.

If you realize LIFE is that way, suddenly high quality isn't all that important anymore, is it? Quality is a luxury and you must have achieved A LOT to be in a position to just think about quality. And the ohter people (management, sales) are not total idiots. If you dream the high quality dream they are the people who pay your bills.

I am an engineer like you, btw. I was just in the situation already, where I had to pay my bills myself and therefore know what amount of humiliation and sweat is required to get money just for ONE person, me.

For example, we recently built a data pipeline that did a bunch of processing and wrote data to a SQL database at the end. For various reasons, there was an unscalable, quick way to implement the write, and a scalable, slow way to do it. We wanted to get the product to testing ASAP, so we chose the quick way initially.

In order to make sure that we could easily replace that code, we ended up creating a separate write function for each table, where the function did nothing else except the write. That involved a lot of duplication, but made it easy to move the tables over to a better method one by one later.

It seems like having functions with one purpose, pure if possible, is a pretty good way to ensure "upgradeable" code–even if the internals of the function are messy, you just have to write a new one that copies the same functionality. Furthermore, I've found single responsibility functions to be easier to enforce in code reviews than single responsibility classes.

Seen this one. It was used as a excuse not to do code reviews because they cause to miss estimates. "Look, the feature is done, but because of the code review requirement I cannot mark it 'done' in the project plan." Solution: abolish code reviews.

* Assuming that good process fixes bad people

This one is a big one. It's everywhere. Especially in big software development companies, such as Microsoft and Google. They tend to believe that once they institute a perfect process, everything works out perfectly. Perfect coding guidelines lead to perfect code, no matter who writes it. Perfect testing process - testing can be done by monkeys. Perfect project management process - now we can hire project managers with just basic Microsoft Excel skills. They don't understand that without actual talent the company enters a "spiral of death" which is impossible to escape.

* Ignoring proven practices such as code reviews and unit testing

This is done frequently by people who never tried such techniques as code reviews and unit testing. If you do it consistently through, say, one release cycle, you start to value those techniques and understand their importance.

Unit testing, for example, helps me to avoid painful debugging complex issues in production. All features that I unit tested usually just work when integrated into the rest of the product. In fact, the last bug I had to fix happened in code that I neglected to unit test, because the unit test setup was too complex for that component (in itself an indirect sign of a problem). Unit tests also lead to components usable independently of each other, thus reducing the overall system coupling.

* Hiring developers with no “people” skills

This is a double edged sword. On one hand a developer who can't communicate well will eventually produce code that doesn't do what's intended. On the other hand there are people with too much "people" skills who can't code shit. They just bullshit their way through. I'd say there are too many of such bullshitters. A lack of communication skills in a developer is a problem fairly isolated to that developer. A lack of coding skills in a bullshitter is much bigger problem that affect many people around him.

My question: can they be truly good programmers of they write bad code? Isn't the essential product of a programmer SLOC and functionality via software? And if they do that in a manner inferior to another dev, isn't that objectively a measure of inferiority in their craft?

I've met far too many 'good' programmers who were a net detriment to a project not to be wary of the term.

ps: i need to tell this more to myself. i already made bad choices regarding where i work, which forced me to quit those jobs at the end..

(edit: spacing)

If you are smart, you will say that you are not interested in having that money if it comes from a loan. If you are not smart, you will say: "wow, this guy is a 10x salesman, we will give him a bonus". Meanwhile the company gets into an unpayable debt beyond any possibility of paying.

Then, when the company is about to die from debt, they declare bankrupcy and start over or sell themselves to be acquired.

Now, imagine it's not salesmen, but software engineers, and it's not actual debt, but technical debt, and it's not bankruptcy but starting your project again.

Ideal code meets both. I'm sure we all agree on that. But when working on deadlines, under pressure, with poor management, you sometimes write bad code. And if the code is 15 years old, that great engineer you are talking about today was, at the time, an inexperienced new coder. So you didn't inherit his greatest work. You inherited his embarrassment where he made mistakes.

I suspect the original authors of the terrible code in question could give much deeper insights into exactly how and why it was done that way.

"Out of the frying pan and into the fire" is not a programming direction I would recommend. (I don't think I've met a single Node developer who isn't bitching about it.)

First, it assumes the developers had substantial control over the schedule for the project ("Giving excessive importance to estimates"). Certainly in my experience this is unusual. More frequently, the schedule is dictated by management, frequently by sales/marketing executives in commercial software development. It is very difficult to push back and a good way to lose your job.

Sales: We have closed this great deal with BigCorp. Can you do X (complicated, challenging software project) by the end of the quarter?

Developers: Err, um, X sounds like a project that will take six months.

Sales: We really need to make our quarterly numbers. Our CEO Bob used to be a developer and he says any competent programmer can do it and we only hire the best. Competent doesn’t cut it here! You are a rockstar ninja, aren’t you? Can you prove you can’t do it by the end of the quarter?

Developers: Well, no. The schedules are driven by some unexpected problem or problems that usually happen. But, well, if nothing unexpected happens, we can do it by the end of the quarter.

Sales: Great! Bob is expecting results by the end of the quarter.

So much for the beautiful, elegant software design methodologies taught in college and university CS programs and peddled by high priced consultants.

Second (“Giving no importance to project knowledge”), high technology employers seem to have extremely high turnover rates of software developers and other employers. Payscale produced a study claiming that the average employee tenure and Amazon and Google is only one year. Many companies seem to target employees with more than seven years of paid work experience — Logan’s Run style — for layoffs and “constructive discharge,” (https://en.wikipedia.org/wiki/Constructive_dismissal) where employees are made uncomfortable and quit “voluntarily.” Undoubtedly, this is costly as the author implies, but it seems to be common practice.

Yes, metrics like “issues closed,” “commits per day,” or “lines of code” don’t work very well. Once employees realize they are being tracked and evaluated on some metric, they have a strong motivation to figure out how to manipulate the metric. Even if the employees don’t try to manipulate the metrics, the metrics all have serious weaknesses and map imperfectly to value added (biz speak).

Third, are code reviews and unit testing proven processes especially for normal non-Microsoft companies? In the early days of Test Driven Development (TDD), Kent Beck and his colleagues made numerous claims about the success of Test Driven Development in the Chrysler Comprehensive Compensation System (C3) payroll project, an attempt to create a unified company wide payroll system for Chrysler. This project in fact had a range of problems and was eventually cancelled by Chrysler in 2000, without replacing the Chrysler payroll systems successfully.

As the problems with C3 have become well documented and well known, TDD enthusiasts have shifted to citing studies at Microsoft and some other gigantic companies that claim practices like TDD and code reviews work well. Are these really true or do these case studies have hidden issues as C3 did?

Further, Microsoft, Google, and other companies that have played a big role in promoting these practices are very unusual companies, phenomenally successful super-unicorns with sales in the range of 40-100 billion (with a B) dollars with near monopoly positions and anomalously high revenues and frequently profits per employee. Microsoft claims to have revenues of $732,224 per employee. Google claims an astonishing $1,154,896 per employee. (http://www.businessinsider.com/top-tech-companies-revenue-pe...) This compares to $100-200,000 per employee for most successful companies.

Fergus Henderson at Google recently published an article on Google’s software engineering practices (https://arxiv.org/abs/1702.01715) with the following statements:

2.11. Frequent rewrites

Most software at Google gets rewritten every few years.

This may seem incredibly costly. Indeed, it does consume a large fraction of Google’s resources.

Note: “incredibly costly”

Companies like Microsoft and Google have enormous resources including monopoly power and can follow practices that are extremely costly and inefficient, which may work for them. Even if these practices are quite harmful, they have the resources to succeed nonetheless — at least for the immediate future, the next five years.

From a business point of view, it may even be in the interests of Microsoft, Google, and other giant near monopolies to promote software development practices that smaller competitors and potential competitors simply can’t afford and that will bankrupt them if adopted.

Both code reviews and unit tests are clearly time consuming up front. Code reviews using tools like Google’s Gerrit or Phabricator (a spin-off from Facebook, another super-unicorn) are committee meetings on every line of code.

Regarding:

Imagine my dismay when I had to collaborate with a colleague on that legacy project and his screen displayed Notepad in its full glory. Using “search” to find methods might have been rad back in the nineties, but these days, refraining from using tools such as modern IDEs, version control and code inspection will set you back tremendously. They are now absolutely required for projects of any size.

Using “search” to find methods was not rad back in the 1990’s. IDE’s and code browsers specifically have been in widespread use since the 1980’s. Turbo Pascal (https://en.wikipedia.org/wiki/Turbo_Pascal) was introduced in 1983 and featured a fully functional IDE, soon to be followed by IDE’s in many other products. Version control dates back at least to SCCS (https://en.wikipedia.org/wiki/Source_Code_Control_System) which was released in 1972. RCS was released in 1981 and version control was common in the 1980s and since.

Code reviews have been around for a long time. However, in the 1990’s and earlier they were restricted to relatively special projects such as the Space Shuttle avionics where very high levels or safety and reliability, far beyond most commercial software, were required. This speaks to the “incredibly costly” quote about Google above.

Without more context, it is difficult to evaluate the use of Notepad. Simple code/text editors like Notepad and vim (formerly vi ) are very fast to start up and can be a better option for some quick projects than starting an IDE.

Some IDE’s are particularly hard to use. Early versions of Apple’s Xcode circa 2010 were particularly difficult to use in practice; it has improved somewhat in the current releases.

People vary significantly. Some developers seem to find stripped down tools like vim or Notepad or Notepad++ (on Windows) a better option than complicated IDE’s. I am more of an emacs or IDE person.

The fact that someone else works differently than you do does not mean they are worse (or better) than you. The fact that something works well for someone else also does not mean it will work well for you — or vice versa.

There are sound reasons for duplicating code, cutting and pasting, rather than creating a function or object called in several location in the code. If the developer anticipates that the code may subsequently diverge, then duplication is often best.

Like grand master chess players, highly experienced developers, especially under tight time constraints (like a chess tournament), code by intuition, not by laboriously reasoning out every step. If it feels like the code is likely to diverge in the future, duplicate. If it does not diverge, no problem, it can be merged back later if needed.

In the bad old days of structured design (1980’s) and object-oriented design (OOD — 1990s), software development projects suffered from Big Design Up Front (BDUF), grandiose attempts to design a perfect software system before writing a line of code. This often resulted in massive cost and schedule overruns and total failures. It often proves better to just throw (“hack”) something together quickly — a prototype, proof of concept, Minimum Viable Product (MVP). Just “get something working.”

Inevitably these prototypes and early stage software projects are going to compare poorly to some theoretical perfectly designed system with 20-20 hindsight. That is what seduced people into BDUF twenty, thirty years ago.

Modern Agile software development methodologies are foolishly trying to have it both ways, have an initial quick iteration BUT that first iteration should be perfectly designed up front — beautiful, elegant, with hundreds of tests, endless committee meetings on coding style and design (code reviews), all sorts of supposed best practices, no code duplication, etc. This is a seductive fantasy doomed to fail in most cases.

"An important component of this project was the focus on deadlines, even to the detriment of code quality. If your developers have to focus on delivering rather than on writing good code, they will eventually have to compensate to make you happy. "

AAAAAAAARGH!!! Seriously? What do you think you're there for?

Good code can be a means but never an end. Of course you're there to deliver. And of course it's more important to do that than to write beautiful code.

Give me strength.

I cannot say this was an issue here, but I can say that some of the worst messes I have seen have followed from this decision. In all such cases in my experience, it was not a technically justifiable decision, and I strongly suspect it was driven by developer ego and overconfidence.

The very worst included a roll-your-own language.

If that's the second thing in the list of problems, the problems aren't nearly so bad or the author's got a savage case of mixing in the trivial with the important.

Then later I come back and rewrite it with a dose of patience. Then my code is much better.

Most production code needs to be revamped every few years because the subsequent unforeseen functionalities that was forced by product managers and customers. That code is usually going against the grain of the original code but you can't blame the original designers because it was never spec'ed out. The best code is the code that is easiest to manipulate and modify, but even then it gets old and needs a rewrite. There's nothing wrong with that, except thinking you can comment on the shitty code and thinking your above it and how you would never let it happen.

Node is how.