Hacker News Reader

Comment by ascendantlogic | Hacker News Reader

I wonder if anyone who wants to bring back the monolith has ever worked on a true large monolithic codebase? Not something that's like 5-10k lines of code excluding frameworks, but monsters that are 100k+. Where test suites take an hour to run end-to-end and you've got 50+ devs all issuing pull requests in the same codebase? I feel like webapps usually have a sweet spot in terms of size and logical reach. This whiplash hype cycle of "X" and "anti-X" just exposes the ever lingering problem of letting blog posts on HN determine your architecture decisions.

View on HN

I've worked on large monoliths (100k LoC is not "large") and have a strong preference for them because I've also worked on large microservice architectures. A monolith with a strong architecture and development process works well. Microservice architectures are often a fig leaf for poor architecture and development process, while introducing new problems and increasing development effort. It is always less expensive in the long run to fix the architecture and process problems than to sweep them under the rug with microservices.

I understand that microservices often become necessary in organizations when they are constitutionally incapable of addressing the root issues directly. But we shouldn't normalize introducing microservices so that we can ignore poor architecture and process, which underlies the majority of cases I see.

At my first job I inherited a 250K LoC monolith which I had to continue to develop and also operate. It wasn't great but I got by. I don't think I could have managed if that 250K LoC had been strewn across a dozen microservices with all the accompanying complexity and extra degrees of freedom in deployment and operation.

SoA is superior when you have large numbers of devs coordinating and working on the same product. For smaller teams monoliths seem to be the way to go.

Sure. Used to be, that's how it was done. Some things sucked about it, others were easier. For example, not much in the way of versioning problems. On the other hand, if somebody broke the release, nobody's new feature made it live. You also had to allow for extra testing time...continuous deployment D is not a great strategy in this environment. We had a week of integration testing for release candidates, unit tests, smoke tests, data integrity tests, UI tests, performance tests, etc. We also had a QA team that signed off on releases. It was actually pretty rare that you collided with other developers because teams were focused on their own areas of the codebase.

Usually codebases are organized into modules/classes/etc and most test suites can run specific test cases (or groups of tests, specific domains, etc).

I think if you can't do that in a codebase, then one has bigger organizational problems than the technology itself.

Also assigning many developers to work on the same set of files is a recipe for disaster (if said developers cannot get along or organize themselves) while working on the very same place/file.

I have. LoC was in the millions. Half a dozen languages too for the different parts of thw monolith. Worked on by a team that peaked at 100 or so. Multi hour build times too. SVN too!

Monolith is still better in my view, despite having experienced that.

The key IMO is to have a clear and singular vision and how all the different parts need to fit into it. If you let each little clump of people do their own thing, then you will have chaos.

Ten years ago I worked on a monster that was half a gig on checkout. Assets, everything. It was an e-commerce site. And it was beautiful. Single set of dependencies, easily tested, one CI/CD pipeline.

It was an egregious Spring/Struts1 app, and it was a breeze to develop on. Fixed my first bug on day 2 of the job. With microservices, would have taken me 3 months.

2G here.

> I wonder if anyone who wants to bring back the monolith has ever worked on a true large monolithic codebase?

Probably not, that's really the point :)

Although I think the limit is significantly higher today. Monorepo tooling is orders of magnitude better than 10 years ago.

Can you recommend something good in that area of monorepo tooling?

I read about Bazel, which looked really nice. Any other interesting tools to check out?

At a certain scale monoliths will no doubt break down but I believe the author and many other developers are bothered when monoliths are an optimal solution for their scale and problem space but end up replaced by one that's suboptimal.

I have as well, and honestly I'd classify something that's 100k+ LOC as a "mega-monolith" vs. what devs would traditionally consider a "monolith"

You don't get to that LOC count without huge teams and decades of time. Any project of that timeline/complexity/size should constantly be having functionality split off so that organizational units can be better groked/understood. To put it a different way, when projects hit this sorta size I have experienced them naturally shedding off functionality that makes sense to be external in a service-based development strategy.

These are the situations where it 100% makes sense for microservice architecture as you've hit the social/technical complexities where breaking things into organizational units makes sense.

I only wish the monoliths my customers were dealing with were tiny 100k LOC projects. I've seen 5MLOC+ nightmares filled with all the terrors you'd expect like unused libraries and dead code that looks should be unreachable, but actually is and creates security issues.

Wait, what? 100k+ LoC is more like a "two-pizza" team and a few years, empirically. And this is in C++, a language where LoC don't flow nearly as quickly as in other languages. Even extremely complex code bases (think database engines) are entirely manageable at this scale and beyond without introducing microservices. Monoliths will scale into millions of LoC with little loss of effectiveness if your code base and processes are setup correctly.

You are describing the consequences of poor architecture and process, not anything intrinsic to software projects.

> empirically

Right, and our empirical observations over these topics are different.

I just read your profile and frankly the projects/industries that we are involved in are very different, only confirmed by throwing "C++" out there - the closest I get to that is some hobbyist C development.

I apologize for agreeing with parent comment's generalization at 100k LOC being a "mega-monolith", along with what I added with team sizes/decades for timelines. Empirically and anecdotally this is what I personally have observed, specifically in regulated fields ie: banking, payments, and education. This is a perfect example as-to what happens when I don't include the word "anecdotal" in my HN comments...

Developers from different industry niches have very different ideals in regards to what a "monolith" is, or what a "two-pizza" team is capable of over time. I find your "wait what?" comment to be a bit dismissive, only to end your comment directly telling me what I am describing which clearly implies I don't know what I'm talking about.

We likely just have different experiences with project complexities, sizes, teams, etc. Sorry if this comes off as aggressive.

I feel like I am going absolutely nuts reading this thread. 10K LoC is 2-3 months of work for a single person hobby project. 100K LoC is maybe a 2-3 person dev team for 2 years. That's assuming a refactored codebase with no copy-paste repetition. "Append only" codebases with heavy copy-pasting can easily get larger than that in shorter amounts of time.

Huge teams and decades of time gets you to 1M+ range, not the 100K+ range.

In a large corporation with tons of people constantly reinventing the wheel for no reason only for resume bullet points... where you spend two months on test specs before putting down a single LOC because "TDD", where you have to work with regulators, auditors, etc...

I feel weird defending myself but... yeah. This is actually my experience.

I get that teams at FAANGs/startups can typically move faster but for your run-of-the-mill corporate America development teams people do not move that fast.

Personally, I have only moved that fast at startups which is why I typically find myself at them.

Again, this is anecdotal to my personal experience.

That makes sense. It just shows how much variation there is across different areas.

Reply on news.ycombinator.com