Hacker News Reader

All of today's software is built on millions of lines of code. This isn't much of a problem when it's hidden away behind a good abstraction. Being written in another language forces the API to be documented well enough so you don't need to go deeper: "native code", "kernel code", "part of the browser".

Crappy million-line Java apps are generally crappy not due to raw line count but rather due to leaky abstractions and badly designed APIs, so you do have to navigate through a lot of that code.

Don't forget that one spring scheduled job that goes rogue and wrecks your sanity constantly.

/mylife

I sometimes wonder if programmers instinctively overcomplicate things in interest of collective job security. Some of the stuff I've seen in (particularly awful) Java code bases is perplexing to the point where it seems intentional.

It's more likely the natural entropy of code - it's easy to add stuff to a system in a way that makes it more messy; and if the system already is a big mess, then it's much harder to do non-messy additions and the bigger mess it is, the harder it is to start cleaning it up.

This is the real answer !

True that. Adding complexity is easy, keeping things at the right level of simplicity is hard work and requires skill. Not just for coding.

I refer to this as the "spaghetti law of attraction". The burden of refactoring things gets higher and higher and no one wants to touch it. So they just add another try-catch block and do some side effect and get the PR merged.

It is also a question of management, it is easier to motivate adding new features instead of keeping thing nice and tidy. Especially as finding good abstractions are very hard and time-consuming.

programmers instinctively overcomplicate things in interest of collective job security

It’s more likely boredom. Cranking out yet another CRUD app is more fun if you do it in a weird way. And a new “skill” for your resume.

Writing clear and as-simple-as-possible code is one of the most difficult parts of programming, one of the hardest skills to learn. In fact, many programmers don't ever learn it. Some do not even realize it's a problem.

I could talk about people unprepared for the jobs they do, and companies and managers setting unrealistic deadlines, having unrealistic expectations, etc. But I think most often than not, the issue is that to really build good code, you need to have a lot of experience and plan and design really well from the start. And simply, most companies and products start as they can. If you start with a deficient codebase, trying to fix that later under heavy budget and time constraints is pretty much impossible, and it's so accepted that some people believe it's the normal way to work.

And even with proper design, clear perspective, and good programmers, you still have to be lucky that no one higher up the ladder imposes decisions that f*ck up all the good work done. There are a few factors more.

What I mean is that you might find all kinds of people, but in general, for those of us who care about what we are doing it's the exact opposite. We consciously try really hard to keep code as simple as possible.

>But the problem is the design and not the philosophy.

If the philosophy prescribes dozens of tools for managing complexity and no tools at all for reducing it than it is the problem.

"Abstractions, decoupling, and modularization of logic/responsibilty" are not some kind of universal good. They are only useful within specific contexts. A lot of software engineers do not understand this and routinely engage in premature abstraction. As a result they produce systems that are 10 times more complicated than they need to be for absolutely no reason.

Java definitely encourages this kind of mentality, because the language itself and its standard library lack in some fundamental areas. Introduction of lambdas and streams helped, a lot, but the overall mentality is still well-entrenched.

I've been hearing the same complaint between Cobol and Java for years : it was simpler before, more efficient, etc.

Of course it was, but you were tied to one system (no application server), security was login/pw, database had no constraint, typing systems were ultra limited, everybody has its own way of writing batches (no Spring), business code was mixed with tons of technical code (no JPA).

Now, sure, if you glue some R, some SQL, etc. you can extract insights worht millions of dollars. But all of that exist just because we have digitalized all of the processes, data collection, etc. And the rise of data scientists will continue only if there are more stuff put in the databases, thanks to you plain, regular, normal programmers...

> But all of that exist just because we have digitalized all of the processes, data collection, etc. And the rise of data scientists will continue only if there are more stuff put in the databases, thanks to you plain, regular, normal programmers...

IMHO there's far to little attention paid to how data might be valuable in an economic sense when storage strategies are being designed by database designers. I recently gave a talk at a developer conference and was really surprised at the level of pushback to adding more data elements or higher precision data "just in case it might be useful".

The preconception that you have to be maximally efficient with storage has led to huge quantities of valuable data being lost.

>> The preconception that you have to be maximally efficient with storage has led to huge quantities of valuable data being lost.

I 100% agree. Many programmers are trained as if memory, CPU are finite resources. Although it's true, in many cases, that reality may be safely ignored, opening tons of opportunities (to store data, to develop faster because you don't optimize, etc.).

At a time where my phone has gigabytes of memory, I'm always surprised that some people ask me to put a limit on a text field. I understand the technicalities behind the question, but from a conceptual point of view, that's often pointless.

The net value of nearly all data is negative. It’s a liability not an asset. Unless you are actually selling data to customers or solving a specific problem that requires collection of specific data, it’s just wasting the company money. You would never hear a manufacturer say “ let’s stock pile aluminum in case we need it”. But that line of thinking comes up all the time with respect to data.

Personally, I am increasingly convinced that a lot of this hate comes from programmers with weak abstract thinking who simply cant do it. Instead of admitting that there is learning cure involved, they will claim the system is bad and everyone else is bad. Compounding factor is difficulty dealing with system that was written by different people who holded different opinions.

Yes, there are badly designed large system. No large system is perfect. However, there are also reasonably designed large systems, including in Java and Java is used in such system for a reason. It is more challenging to write large system. Yes, it is harder when parts of system are written in style that was considered best practice few years ago but was abandoned since then.

If you are spending a lot of time looking at millions of lines of Java that do almost nothing, them you likely dont really know what it what and need to read up more. At least that is my experience.

Badly designed systems aside, for most systems that currently exist or have existed in the past, there is little or no documentation that is worthy of existing. Most comments in code and the associated documentation in manuals fails to provide the reasoning as to why the code exists, why it is written that way, what underlying assumptions have been made, etc, etc, etc.

I am going through a process at the moment of documenting all of my local codebase. It will, in turn, be turned into a literate programming base. The problem I am finding is understanding all of the assumptions that underlay the original code. Why was it written this way or that, what is it trying to do, is the code actually doing what it is supposed to do?

There are, at present, some questions that I am having difficulty answering and this is my code. How much more difficult is it for someone to come in and look at a historical piece of code and follow what the original authors and designers were trying to achieve and what were the changes that have been made over time trying to achieve.

Documentation at the level we need to be able to adequately maintain any code base is just not done - it is very hard to do and to do so in a way that will help future people manage and maintain that codebase. On of my projects involves restructuring the code base. However, I need to understand the history of that codebase and that means talking with those people who are still living who knew the original authors and give an oral insight as to why things have been done the way they are. This oral history has to be written down and the codebase documented with it. When that information is in place, why the code is written the way and how we can now rewrite that code to be more effective is now achievable.

If we then put on top of missing history and documentation all of the bad designs, well, we are then facing even bigger problems. Then we put on top of that all the egos and politics involved, we get an even bigger mess.

So just reading up more doesn't actually help, because that which is needed was never written in the first place.

I meant documentation on whatever tools and libraries project uses. Specifically, a lot of lines that do nothing are 95% result of frameworks integration.

You should not need to look at those particular lines that much.

I agree that the other issues you described here make work on large long running projects harder. It is challenge and somethings fight for every inch. Which is why I am increasingly sick of people who can't do it kicking those who can and kicking tools that make it possible (I have better chance to figure out the system with oral history only you just described then in javascript or python. Not easy, but tools make it less hard.)

The problem isn't the abstractions, it's the sheer size of the codebase. The author - and I think most people - prefer a codebase they can grok. Nobody can grok millions of LOC, at best they can have a high level overview of what does what.

At that point you need the abstractions and practices that make code boring.

It would be far from the first HN article (or comment) that failed to make a distinction between "poorly designed object-oriented systems" and the very idea of OOP, design patterns, etc

nit: is it "about poorly-designed, object-oriented systems"?

no.

It's "Something about poorly designed, object-oriented systems."

1. compound adjectives are hyphenated, except when the first world ends in 'ly.' 2. two adjectives, should have comma, similar to 'big, green systems.'