Hacker News Reader

Comment by mbork_pl | Hacker News Reader

> It's okay to have files with 10,000 lines.

Ever since my time as a mathematician (I worked at a university) and using LaTeX extensively, I never understood the "divide your documents/code into many small files" mantra. With tools like grep (and its editor equivalents), jumping to definition, ripgrep et al., I have little problem working with files spanning thousands of lines. And yet I keep hearing that I should divide my big files into many smaller ones.

Why, really?

View on HN

I think what goes wrong with dividing is people truly just splitting the code into multiple files.

I think the code should be split conceptually thus not just copy/paste part of code from main file to submodules, but split the code around some functional bounderies or concepts so that each file is doing one thing and compose those concepts into more abstract concepts.

So that when I try to debug something I can decide where I want to zoom in and thus be able to quickly identify the needed files.

I think the big benefit is not the actual split into files, but the coincidental (driven both by features of some languages and also mere programmer convenience) separation of concerns, somewhat limiting the interaction between these different files.

If some grouping of functions or classes is split out in a separate file where the externally used 'interface' is just a fraction of these functions or classes, and the rest are used only internally within the file, then this segmentation makes the system easier to understand.

One big reason is source control. Having many smaller files with well defined purpose reduces the number of edit collisions (merges) when working in teams.

Also, filenames and directories tree act as metadata to help create a mental map of the application. The filesystem is generally well represented in exploratory tools like file browser and IDE. While the same information can be encoded within the structure of a single file, one needs an editor that can parse and index the format, which may not be installed on every system.

Well, Git is pretty good at merging when the changes are in different places of the same file. Though your second point is a very good one.

FWIW, sometimes when I worked on a really large file, I put some ^L's ("new page" characters) between logical sections, so that I could utilize Emacs' page movement commands.

It's not just git, it's pull requests and code reviews that block on concurrent file updates. The workflow friction multiplies with the number of devs sharing the same code area. Small files minimize this by making more granular work units.

> Also, filenames and directories tree act as metadata to help create a mental map of the application

Correct. But this is an argument against splitting: once your folder structure reflects your mental model, you should no longer split, no matter how big individual files get. Splitting further will cause you to deviate from your mental model.

Also, it seems like we're arguing against a strawman: saying "big files are okay" is not the same as "you should only have big files". When people mean is that having a big file does not provide enough justification to split it. But it is still a signal.

At the risk of getting lost in a swamp of not particularly good answers, it's most useful if you have scope control: If you have language keywords that allow you to designate a function as "The context/scope of this function never escapes this file," then multiple files suddenly become very useful, because as a reader you get strong guarantees enforced by a compiler, and a much easier time understanding context. The same can be said of variables and even struct fields. In very large programs it can also be useful to say, "The scope of this function/variable/etc never escapes this directory".

If everything is effectively global to begin with, you're right, it might as well all be in one file. In very large programs the lack of scope control is going to be significant problem either way.

This is where object-oriented programming yields most of its actual value - scope control.

This is one of alarmingly few sane, articulate comments amid an extremely weird conversation about how maybe a giant mess of spaghetti is good, actually.

Yeah sure overengineering is a thing, but you're way off the path if you're brushing aside basic modularization.

Because one day someone else may need to read and understand your code?

Mathematicians rarely collaborate by forking and improving other papers. They rewrite from scratch, because communicating ideas isn't considered the important part, getting credit for a new paper is.

Fair enough, though I've been working as a programmer for over 6 years now (and I've been programming on and off, as a hobby, for over 3 decades).

But is it generally easier to read and understand, say 10 files of 1000 lines or 100 files of 100 lines or 1000 files of 10 lines, compared to one 10,000 lines file? (I don't know the answer, and don't have any strong opinion on this.)

navigating between files is trivial in most real IDE's. i can just click through a method and then go back in 1 click

But navigating between functions / classes / paragraphs / sections is also trivial in most real editors.

But this is precisely my question: in what way does splitting the code into many small files help with that? Personally I find jumping between many files (especially when they are located in various directories on various levels in the fs) pretty annoying...

Of course, that probably depends on the nature of the project. Having all backend code of a web app in one file and all the frontend code in another would be very inconvenient. OTOH, I have no problems with having e.g. multiple React components in one file...

Obviously you can split your code into many files in a way that obfuscates the workings of the program pretty much comoletely. And I think you can write a single 10kloc file that is so well organized that it is easy to read. I just never have seen one...

I believe that files offer one useful level of compartmentalizing code that makes other people easier to understand what is going on by just looking at the file structure before opening a single file. The other guy can't grep anything before they have opened at least one file and found some interesting function/variable.

I often want to have multiple parts of the code open at once. Sometimes 5-10 different parts of the code (usually not so many as that, but it depends what I'm doing) so I can flip between them and compare code or copy and paste. Most editors I've used don't have good support for that within a single file (and having 5 tabs with the same name isn't going to make it very easy to tell which is which).

Very good point, though this is really a deficiency of "most editors". ATM I have 300+ files open in my Emacs (which is no wonder given that I work on several projects, and my current Emacs uptime is over 6 days, which is even fairly short). Using tabs is bad enough with 5+ files, tabs with 300+ would be a nightmare.

That said, Emacs can show different parts of the same file in several buffers, a feature which I use quite a bit (not every day, but still). And of course I can rename them to something mnemonic (buffer name ≠ file name). So I personally don't find this convincing at all.

Because often you don't know what to grep for, or search is too general and returns lots of irrelevant results, or perhaps you're just in process of onboaring on the new project and you want just to browse the code and follow different logical paths back and forth...

and when dealing with the code that's well-organized and grouped into logically named files and dirs, you simply can navigate down the path and when you open a file all related code is there in one place without extra 10k lines of misc. code noise.

Just one of many reasons: parallel compilation.

Another excellent point, but only applicable to compiled languages (so still not my case).

Just two of many reasons: parallel linting, parallel parsing.

Ah. Silly me. Still, linting/parsing 5000 sloc is almost instantaneous even on my pretty old laptop.

5000 LoC isn't that much.

There are many other reasons why separation is better for humans (separation is organization) but these arguments about parallelism are valid for androids, humanoids and aliens.

It is a kind of cult really. Along with the rise of modern editors which somehow craps out at relatively large files. So small files are sold as well organized, modular, logically arranged codebase. You see these lot of adjectives to support short files. None of them seem to be obviously true to me.

Generally speaking those 10k lines files are shit code, regardless of context

Generally speaking it's only Enterprise(tm) code that has miniscule (not small, miniscule) files with shit code

As long as it's good code, no one cares. If you aren't working on Enterprise(tm) code odds are every bad code you'll look at has way too many lines in the abstraction unit that is being used

If you change 1 .c file, your compiler needs to recompile the .o file for that file and link it with the unchanged .o files.

If you have everything into one .c file, you need to recompile the whole thing every change.

If the file only contains static methods, it doesn't matter too much. However a class with mutable state should be kept pretty small imo.

This only applies to OOP, no?

Or module level state (Go, Python) which is in many ways even worse

grep can search multiple files. Tags can jump between files.

LaTeX is an obvious case: why pay to recompile the whole huge project, for each tiny change?

Actually, LaTeX has to recompile everything even after a tiny change (there are rare exceptions like TikZ, which can cache pictures, but even then it can put them into some file on its own AFAIR, so that I can still have one big file).

Now that I think about it, LaTeX is a very bad analogy anyway, since LaTeX documents usually have a very linear structure.

Reply on news.ycombinator.com