Architecture.md (2021)

I like this idea but IMHO, regardless of repo size, architecture can still has some place in a Readme. For example, I purposely placed a Mermaid sequence diagram[1] in the main Readme because I think it's important that all readers see and understand its workflow[2]

[1] https://mermaid.js.org/syntax/sequenceDiagram.html

[2] https://github.com/hbcondo/revenut-app?tab=readme-ov-file#-w...

Here here! This sounds like such awesome advice.

I wish we had better tools to visualize architecture of running systems. It's crazy to me that reading the code or a markdown file are still so state of the art. Maybe if someone's fancy they'll have some nice Mermaid diagrams. I want the architecture to be able to show itself live. Broadscale macroscale observability, baked in.

I think this would help everyone be able to appreciate & grok computing much more, would help humanity augment itself.

See https://adr.github.io/

https://github.com/shipmight/shipmight/blob/master/src/ARCHI...

At the top of each file there was a tree of links to other ARCHITECTURE.md-files in the repo, like this:

    * ARCHITECTURE.md <- you are here
    * backend/ARCHITECTURE.md
        * backend/api/ARCHITECTURE.md
        * backend/cli/ARCHITECTURE.md
        * backend/ui/ARCHITECTURE.md
        * backend/utils/ARCHITECTURE.md
    * frontend/ARCHITECTURE.md
    * internal-charts/ARCHITECTURE.md

  the shorter it is, the less likely it will be invalidated by some future change. This is the main rule of thumb for ARCHITECTURE — only specify things that are unlikely to frequently change. Don’t try to keep it synchronized with code.

I agree it is the difficulty in grokking a codebase. "Pattern" naming sort of helps, but I end up having to read a far bit.

On github, I always keep thinking the commit messages for each file are descriptions. Would that be more useful?

Now I'm surprised how uncommon this is in open source.

I've added architecture files to projects at multiple jobs now [0], [1] and they've been well received. They're not perfect, but they're better than nothing.

[0]: https://github.com/zapier/zapier-platform/pull/324

[1]: https://github.com/stripe/stripe-cli/blob/master/ARCHITECTUR...

  - README-driven development
  - ARCHITECTURE.md
  - ADRs
  - arc42
  - C4
  - etc.

Instead of using somebody else’s standard, I just organize and refactor docs as I go, in the same way as I manage my personal notes in Obsidian.

Initially I wanted to use a common subset of Markdown that will work both in GitHub (GFM) and Obsidian, but then I gave up, and just use Obsidian flavor of markdown with all its proprietary features like Dataview plugin, templates, etc.

Mermaid and LaTeX is built-in in Obsidian, and there is a plugin for PlantUML.

For visual drawings/diagrams there are builtin Canvas, DrawIO and Excalidraw.

Architecture.md - https://news.ycombinator.com/item?id=26048784 - Feb 2021 (153 comments)

A problem, though, is that many things lack an architecture; they were grown and are a mess, and the procedural knowledge can't be put to a document without just pointlessly recapitulating the code itself in what is possibly a more confusing form.

I developed the following rules, which can be automatically enforced if you explicitly write the ranks (don't try implicit ranking; that means you won't get sane errors when you violate the rules):

0. Every directory has an implicit dependency on all its contained children.

1. A directory that does not depend on any other directories has rank 0. It suffices to only use rank between sibling directories, and it's probably simplest to maintain, though global ranking does work.

2. A directory that depends on others has a rank of 1 + the highest rank among its dependencies. To ease refactoring you could loosen this to "has a rank that is strictly greater than the highest rank among its dependencies".

3. Thus, circular dependencies between directories are forbidden; refactor (preferably, by splitting directories; most projects are too merged already) until you have a DAG. (circular dependencies between files in a single directory are allowed, subject to language-specific caution)

4. Thus, it is forbidden for a subdirectory to depend on a parent (or ancestor). If you encounter this, move the relevant files to a new subdirectory (since depending on a sibling or uncle is okay; if rank only applies between siblings this means the parent has to add a dependency on the uncle so that it gets the correct rank).

5. Depending on a cousin (or nephew) directory should be treated as a dependency on that cousin's parent, though (depending on what amount of directory structure your language forces you to use) it may be a hint you're doing something wrong.

6. Each directory can produce at most one library (shared and static count as the same library) or executable (at least, user-facing ones; code generators and tests might not count). Note that for other reasons it's generally inadvisable to ship multiple shared libraries or multiple static libraries, though you might use them during development.

7. (YMMV) If a directory contains any generated files (or their inputs), it should not contain any other files. Note that there are at least 3 major workflows for generated files, so the details will vary, but isolating them is useful regardless.

Note again: this is both coarser and finer than build dependencies - we treat directories as units, but add conceptual dependencies. As a general rule, I find it useful to define that the client (which calls `connect`) depends on the server (which calls `listen` and `accept`), and/or the data consumer depends on the data producer. Admittedly I have not deeply considered the case of servers that are worker-like, but note that it is often still possible to satisfy both by splitting directories further.

Note that additionally defining a "weight" (1 + weight of dependencies), although possible, is not particularly useful at the directory level. Long chains are easier to understand than tangled messes, but have a higher weight, and we don't want to discourage splitting a directory into a chain.

=====

In my experience, a project of about 100 kLoC organized itself about 20 immediate subdirectories of src/ with a max rank of about 10. This was roughly follows:

0-1: core support for the language, compiler, and replacements for the standard library; these almost never change. If somehow you have other kinds of code that doesn't depend on these, consider artificially inflating their rank to at least 2. About 1 each.

2-3: most fairly-project-agnostic (but not polyfill-like) "common"-like directories. About 2-3 each, probably.

4-6: semi-project-specific common stuff, most executables/libraries. IME generated files tend to belong here, and may be responsible for splitting a directory into a chain of length 3. There are a lot of these, usually with 1-2 dependencies of the previous rank and several more of the rank before that; likely there is nothing that depends on the entirety of any previous rank.

7-9: dependencies of the most complicated executable. About 1 each, a single chain at this point, though they might still have low-rank dependencies. At this point there's a decent chance that a directory might depend on the entirety of rank 2.

10: the most complicated executable (at least for me, this was the executable that had a conceptual dependency on other executables, even if it didn't have a source dependency. If you have a simple executable that conceptually depends on a complicated one, the simple one would be the highest rank instead). Note that only one of its dependencies is a 9; the rest are in the 3-6 range.

An alternate structure with deeper nesting, which I considered but never bothered to implement:

  src/base/ - the directories ranked 0-1 above
  src/lib/ - everything that either is part of a shared library, or is used by multiple executables (all directories ranked 2-3, and some ranked 4-6)
  src/each-executable/various/ - dependencies of a particular executable (including all directories ranked 7-9 above)
  src/each-executable/main/ or just src/each-executable/ - the file containing `main` and as little else as possible (otherwise we're likely to confuse "code closely related to `main`" and "code that is executable-specific but not involved in many dependencies")

0: scripts/ - executable scripts used during the build that don't have to be built, or that might be installed with no more than a shebang update.

1: src/ - all source code, including that for tests and tool/

2: tool/ - built executables needed for later parts of the build. Refactoring to split this out from shebang scripts is nice for your sanity, but may be noisy, especially if your build system's dependencies are sloppy. In contrast to build/ these are never cross-compiled.

3: build/ (all other output (data and potentially-cross-compiled code); contains bin/, lib/, and share/ at least)

Hopefully, projects that get much bigger than this can be split into further subdirectories so that the dependency ranking only need be done separately within them. I'm not sure if any big project is actually that nice in whole, but you should at least be able to create sanity in part without too much distress.

That said, keep in mind that this is just one approach to this particular problem, and it is just one architecture-related problem. Particularly, if your tooling makes it difficult to split directories (or files for that matter), fix your tooling first (related: "recursive make considered harmful")

What about aspiring to "screaming architecture" instead? Don't hide your application domain in a "crates" directory. Do it the other way around.

ASCII diagrams of components or your draw.io diagrams would go here. Knowing what you have is half the battle.

Compass cause it helped you navigate. It's short. Every 'module' needed to have such a file. It was also enforced in the CI.

I agree with this. A map of the code is great. It's like an exploded-view drawing of a mechanical component. It helps highlight what goes where and connects how to what.

...but is this what "architecture" means? I was under the impression that architecture went beyond major-units-and-their-interfaces and had more to do with the decisions and assumptions that lead to those specific units and interfaces -- the why behind the what.

In other words, the "architecture", in my view, is the thing we might go against when we refactor things. Not because names and interfaces change, but because the rationale for having things a certain way might still apply only we don't know about it because those assumptions and decisions -- the architecture -- is rarely documented and still would not be under this proposal.