The creeping scourge of tooling config files in project root directories

I don't get this, what's the problem here? Having config files in root means you know they can apply to all sub directories. This has nice properties like recursive behaviours (see .gitignore)

If you want to put (most of) your config files in a .config folder, then most apps should support that

If you want to view your config files in a different way, then actually you're trying to solve a different problem. Perhaps hide show toggling or automatic grouping (see macOS desktop stacks) with your fs viewer would be a better area to tackle.

Finally, just wanted to say, everything at root level is usually config or a folder, so if I want to understand what tooling a project uses, I know immediately where to go, code is always in a "src" or similar. It sounds like this is a feature, not a bug.

I 100% agree on this. But it's been pretty hilarious on how obstinate tool makers are about changing to Python's new pyproject.toml [1]

Python packaging is a bit of a mess already, so when I recently started a new small package I wanted to choose tooling that would not clutter, but a fair amount of tool makers were reluctant to allow code into their repo that would use the unified toml rather than a ton of separate file.

.config is probably a better solution than a single top level toml, IMHO, but far more important is doing something unified rather than continuing the pollution of the top-level namespace, which merely obscures the project structure.

[1] https://snarky.ca/what-the-heck-is-pyproject-toml/

The amount of pressure you feel telling you you are using too many tools should scale linearly with the number of tools you use. This discomfort you feel with seeing all of those config files is telling you something. Use fewer tools.

I'm not sure why this [1] pattern can't be adopted/extended. Put everything in a .config directory.

Search path for config is:

* /etc/config

* $XDG_CONFIG_HOME (~/.config)

* $PROJECT_CONFIG ($PROJECT_ROOT/.config)

* directories between $PROJECT_ROOT and $CWD for .config

* $CWD/.config

This allows for system wide, then user wide, then project wide, then directory hierarchy configs, with lower levels overriding higher ones.

Inside .config, each tool can have it's own chosen file, directory, extensions directory, so:

.config/$TOOL.{json,yaml,toml,ini,xyz}

.config/$TOOL/config.ext

.config/$TOOL.d/modular config files eg 00-module.ext

Most of the time for most projects, there will be system wide and user wide config, then a project wide config. Overrides lower in the hierarchy tend to be less used.

Yes, it means that there's a tree search for a tool to get its final config. Whether that's a real issue for performance or otherwise is a potential to remove some of the hierarchy/search.

[1] https://specifications.freedesktop.org/basedir-spec/basedir-...

How conscientious.

And yup, conventions like this work well Linux and BSD's. To note ~/.config for many configs

These days you see this happening inside of .github/, where configs related to gh repos and actions go.

If tooling authors started universally recognizing .config/ as a directory where we could keep stuff, the root could be super clean.

How about just one file, across languages / CI tools / etc? a "project.toml" or "project.yaml" or "project.json"?

    [tool.npm]
    name = "frontend"
    version = "1.0.2"

    [tool.npm.dependencies]
    react = "^16.3.1"

    [tool.travis-ci]
    # ...

    [tool.eslint]
    # ...

    [tool.poetry]
    name = "backend"

    [tool.poetry.dependencies]
    django = "~3.1.0"

If vendors were willing to accept it, it'd be less clutter. Integration tools would only need to check one place (legacy configs would still have to be supported, though, so it's still a dream)

Legacy configs could be ported via YAML / JSON straight into the TOML sections. It'd work for package.json or tslint.json, but not stuff involving runtime, e.g. .eslintrc.js

I think ultimately it comes down to personalities. I'm the kind of person that likes directories and folders. And having top level files feels clunky and disorganized. Like much organization, taken to the extreme it can be a form of procrastination to avoid doing real work. Yak shaving, if you will.

But opening the root project folder to just a list of subfolders gives me a very nice first impression.

Below is my list of config files for https://www.joyapp.com (Django + NodeJS). Not a hugely complex architecture, mind you.

Some notes: .editorconfig at least tries to consolidate this for some things. And pyproject.toml could help with Python. Still, definitely jarring and something I've noticed.

    .coveragerc

    .dockerignore

    .editorconfig

    .flake8

    .gitattributes

    .gitignore

    .isort.cfg

    .prettierrc

    .shellcheckrc

    jest.config.js

    mypy.ini

    package.json

    pytest.ini

    requirements.txt

    shell.nix

    tsconfig.json

    tslint.json

    webpack.config.ts

    yarn.lock

On another note can we talk about the amount of projects that just dump their config/log files in the root directory as a 'hidden' file when you start them? Why? we have a .config for a reason.

Ehh, this is just moving the mess from one location to another. All these config files can be grouped into two or three conceptual things:

* Dependencies - Things required for your code to run, for example Dockerfile, package.json, yarn.lock

* Metrics - Measuring code quality, but the project works without these, such as jest.config.js, .eslintrc, .eslintignore

* Other - I don't recognize several config files here, and suspect some don't fall into the categories above, but am not sure. Also, README.md and .gitignore would fall into this category, but probably ought to remain at the top.

So instead of a generic "configs/" directory, if I was to organize these myself, I'd probably want at least two subdirectories so it would, y'know, actually be a bit more organized.

To me this is part of a larger problem: tools that expect a given directory layout, or even worse, that they are the sole tenant of a given repository. Just because my project contains some node.js code doesn't mean that I want to have a top-level, global node_modules directory.

Let me organize my repository files however I want and however best fits my project organization.

I don’t find it a scourge. I actually find that I can look quickly at a project and get an idea of the tools that they are using. This seems to be a bonus and really a few more files isn’t a big deal.

This is a UI problem, not a file location problem. GitHub and VS Code (both M$), could implement a nice way of grouping these files out of the of other files at the tip level.

I feel like a given tool deserves at best a single item in the root directory. If your tool needs more than one, you should be making a folder. I'd be totally on board with putting all those folders in another folder, but I'm iffy on the indeed the frustration of getting everyone to agree on one.

My head hurt thinking about how much efforts and coordination across multiple ecosystems required to pull it off. And if they do manage to pull it off, the reward is... slightly cleaner project top directory?

Fully agreed but if everything is in one folder out of the it doesn’t need to be hidden anymore, it could be “config” instead of “.config”. Maybe even “config” (project config inside version control) and “.user-config” (project config by developer not inside version control). The issue hints at this but deliberately descopes user configs. Files inside the folders could have a dot depending on if they are normally edited throughout development or normally left alone.

It's not a scourge. It's a sensible default location. Most tools let you explicitely specify a config file path if you want to store it elsewhere.

Also, please don't use dotfiles. There is no reason to hide such important files.

I mean, for most tools it would be pretty easy to look at .config/ first and then at the project root - and print something about the new best practice.

The whole thing is a little uncomfortable though. There is a clear isomorphism between the contents of the structure of, say, a JSON file, and the contents of a directory of structured text files (and other directories). Heck, you could build a little tool with a slider that moves all the data between a single file, and a deep directory hierarchy with lots of tiny text files as the leaves.

Personally I feel like the best solution is to somehow "indelibly" combine the tool with its config within a project, de facto removing all config options from the developer. In the worst case, it could be something as silly as a wrapper script with config embedded in there. Of course this is a kind of fiction, but a useful one. It feels nice to get to a point where you don't have to think about tooling. (The last time for me was like 2000).

This is a non-issue, thus no need for a solution.

If you have too many config files, then you should use less tools.

Besides, most of them are already hidden anyway (because of dotfiles).

This is fine. It's practical, but a little ugly. Same discussion that we had the other day with Tailwind CSS. It's fine, it's very practical, but a little ugly.

Who cares? The obsession with hierarchy is silly. Good search beats hierarchy.

I think a good fix would be hiding dotfiles on repository view like on GitHub. "ls" hides dotfiles from you by default, so why shouldn't GitHub. You can always add a checkbox "Show hidden files" or instead of completely hiding them put them in an accordion (12 hidden files) that people can click to reveal them.

I'm not a node dev, but our projects at work keep growing with helper files. We have a .buildkite to control the build, we have a makefile, a Dockerfile, a docker-compose file, go.mod and go.sum. These core files are about half of what we had a few years ago, so that is nice. The makefile references a dozen helper scripts though that are also checked into the repo (helpers to run tests, build images, publish images, clean up, etc). Most of these are vastly similar between different repos/services we run. I can't think of something better, and there is a part of me that enjoys the lack of magic: just follow the code starting at ./buildkite which will point to each make operation which will point to each helper script. The other part of me feels like, "I just want to write some code and as long as my project is structured right, everything else should be shared tooling _outside_ my repo."

It's probably too late to change every tool.

If the main problem is that, when you go look at a project on github or gitlab, you see a long list of config files rather than the code you're looking for, a cleaner solution is for the git sites to show the listing for a /src directory instead of / on the project page.

The root is the right place for project-wide settings.

GitHub and other tools should do what ls does and hide dotfiles by default. That’s why they are dotfiles.

Other project-unrelated config/metadata should follow the .git example and use dotfile config files, preferably yaml or something similar (not json, as it does not support comments).

The suggestion in the thread to "support defining this in package.json" (https://github.com/nodejs/tooling/issues/79#issuecomment-664...) reminded me of autoconf. It also has the issue of lots of auxiliary files in the project root (but worse since they're not hidden files), and the solution there is that you can add "AC_CONFIG_AUX_DIR(aux)" to the configure.ac to make it look for these auxiliary files in the "aux" subdirectory instead of the project root (for instance, "aux/config.guess" instead of "config.guess").

I like how Gradle works.

Basically, if you want to add a new "tool" to your project, you add it as a plugin in `build.gradle`:

    plugins {
      id "com.github.spotbugs" version "4.5.0"
    }

Then, in the same file, you configure it:

    spotbugs {
        visitors = [ 'FindSqlInjection', 'SwitchFallthrough' ]
        // more config
    }

Some plugins require external files, but that's usually because the plugin was not designed for Gradle and/or the authors didn't bother to add some code in their plugins to read config from the project file (which is usually very easy).

I feel like the is just unecessary OCD behavior. We get enough of that in software already

The good old "conventions" vs "configuration" debate. Seems like Javascript looks more and more like Java with all these manifests.

Though XML solved the issue of needing multiple config files long ago, with namespaces.

If the organization of files in your repo is your biggest problem, you've obviously got excellent code and no tech debt.

In which case you probably have enough free time to build a tool that just moves files around at runtime. Organize your repo however you want, then run your tool as a wrapper for any other command, and it'll move all files into the places other tools expect them. Now you've solved the problem, which was that humans like aesthetically pleasing things that serve no useful purpose.

There’s a solution to this problem: https://github.com/sindresorhus/hide-files-on-github

Additionally, your OS probably already natively hides files that start with a dot, so this is just a UI problem.

Please don’t “solve” this issue by moving files to a sub-directory. If anything, only leave non-config files there, it’s the obvious simple solution that most projects follow anyway.

This is a problem I've encountered on my projects. I just want to nestle them all in a `config/` folder. Alas, lots of the tools really dislike if you would like a configuration in a spot other than the root directory. Suddenly you get to enjoy debugging all the times some functionality wants to be relative to the config directory instead.

I love the idea of a standard config file, but... Cross-project standardization in the JS ecosystem? Unlikely.

The Rails community attempted to do something like this with the introduction of Webpacker in Rails 5. They backtracked after a few months. Most likely after realizing how many thousands of npm packages make tightly coupled assumptions to the location of things like ‘package.json’ and the amount of patches it would require.

In the screenshot in the article, 5 out of 8 are related to using JavaScript. Anecdotally, it does seem like more of a problem with Docker + JavaScript + Git in the same project.

I know some tools allow you to specify the location of your config files, and it would be nice if all tools started to do that.

Rather than a .config/ folder how bout a .tooling/ folder ... as it is for the tooling. Projects usually have a config dir for server / environment configs. Would be easy to confuse the .config/ of tooling with the config/ for the app itself

I would assume all these tools to support a —-config or —-file parameter to specify where my config is. Couldn’t that be specified in package.json directly? That way you can move your config wherever you want.

This is a problem mainly for people who use GUIs.

When you use a terminal you don't really care how many files are in a directory, unless they have names that are annoying to autocomplete.

This is not a new problem. You should see my home folder. I hardly ever look at it, I moved into ~/Downloads since at least I can keep that tidy myself.

Interesting discussion.

If it's about "cleaning up" before committing, a single script that moves files back and forth from project root to a .config path might also be an option...

I'm using .config when I find it but for the last 15+ years on new projects, regardless of language I'd been making an `etc` directory and putting config in there.

what's the difference between /tool.cfg and /tool/tool.cfg?

if your project is a hodgepodge of tooling the problem is the hodgepodge of tooling in your project, maybe you need to nest modules or to split libraries, instead of changing everyone else sane defaults because modern hip toolchains can't handle dependents and nested subprojects without stepping on each other toes or requiring the full lib in a single place

The great thing about all these config files is as a barometer for the popularity of various bad flavor-of-the-month config file formats reinventing key=value.

I find many developers have an aversion to directories containing many files, even if those files semantically “belong there“.

I think our tools drive this aversion quite a bit.

A nice convention in python projects is that you can use setup.cfg or pyproject.toml for many tools and they each just use their particular section.

Simple. Add a new config to the project root directory that defines the config locations. Done. /jk

I do like the .config directory naming suggestion though.

This is not what unexpected!

To me the scourge is tool config files being in the repo at all.

Ok I’ll make an exception for .gitignore or something that enforces house style or formatting conventions to make things proper for check in.

But an editor config? That’s so user-specific and shouldn’t affect (or should I say “infect”) the source code.

I put my noise in an etc/ subdirectory.

How come /usr/etc/ didn't become popular?

I also personally think it sucks.

Put everything in a folder, and use a namespace, e.g.:

    org.organization-name.project.json