Interesting Codebases

I've actually done undergraduate research with AOSA, and by far it was the most interesting to analyze the differences and commonalities in architectural approaches between FOSS projects with similar domains.

I'm with you on this! Clicked on the same suggestion and had no idea where to start. Expected to see a "src" folder which is typically used to separate the source code of the project from the docs/objs/tests/utils, it was missing so I closed the tab not willing to search for each part of the actual code was.

For what it’s worth, this is how I usually go about exploring codebases ( I am the author of that blog post ).

I either have something specific in mind that I want to understand, e.g in the case of Seastar, I wanted to understand their reactor design and implementation, so I locate the respective file(s) that implement them and I start from there, and then I just branch out to other files -- I usually keep multiple vim windows open, and I take notes.

When I am not looking for a specific answer, I choose a directory, and then I sort its files by size (e.g ls -lShr *.{h,cpp,hh,cc,java} ). I usually sort by file size in descending order(smallest first), but some times it makes more sense to sort by largest file first, and I start from there. I still map my way around, and if something stands out, I open the respective file in another vim window/tab, and look it up, and then continue with the previous file.

Search for "Code Review" posts by the author of the "Game Engine Black Book":

http://fabiensanglard.net/

He walks you through his whole code review process starting from cloning/building the project:

http://fabiensanglard.net/doom3/index.php

http://fabiensanglard.net/quake2/index.php

I start with finding entry points and analyzing data structures.

Generally, if it's an executable, you should look for the main function. Otherwise, it's a library, in which case you should look at the documentation and start from the handful of points of entries into the library.

When in doubt,

    grep -r -- 'main[ ]*(' .

If the project is using git, there's a git-extras package to show statistics on the git history where you can see the files that are most frequently worked on.

https://github.com/tj/git-extras/blob/master/bin/git-effort

First and foremost, clone and build it. If you cannot clone, build and get it running within an hour, it's generally crap and not worth your time[0].

Once you get it building and running, you can now start making simple changes in order to find your way around.

[0] Developers should pride themselves on how (relatively)simple this process is for their project(s).

You're right, such things should actually be in the README, but go figure, nobody does it.

Documentation should not be a matter of "every function has a docstring", a description of the architecture is always useful.

The question to me is, do you really learn anything just by reading it or is it just an exercise in self-aggrandizement and programmer posturing. You will be able to say: I've read that code and understood it. Some people claim that it's more important that you also look at the "bad" code and find ways to improve it that way you'll be truly learning the craft of programming and not just be on board with the latest engineering practices of a particular language and tool-sets.

An important thing it has, is an actual source code layout in the readme, something that is missing from nearly all open sourced projects, making it a hassle to map out the project and start reading them...

This is a fantastic example; thanks for sharing!

I never implied that I studied the Linux kernel codebase insight out -- specifically, I was interested in the networking stack and the VFS layer, and from there I looked into the memory subsystem and certain drivers implementation. I did map the codebase to the extent that I can more less find my way around easily.

Same is true for Chrome. I was interested in the code for the various UI components, but I looked into other bits here and there.

I 've studied most of Postgres, LLVM and Tensorflow codebases - as in, I went through pretty much most if not all files looking for interesting bits (and finding plenty). I guess there's no way to "prove" anything to anyone, but I don't really care or want to do that either -- it's not about bragging rights, if that's what you implied; I just thought I 'd share a list of codebases I came across that I thought were interesting and worth of other people times.

As for my job, and my work, you may want to check out my Github profile (https://github.com/markpapadakis) -- though there's only public stuff there. I guess I like to spend my free time learning instead of say, watching tv or waste time elsewhere:)

At least in regard to the Linux kernel, the interesting thing about this code base is that you can study the different sections separately. For instance the networking code, the VFS, the scheduler, are quite isolated from other stuff. However this also means that there is a variety of authors writing a variety of unrelated parts, in practical terms is a non uniform code base so it's hard to have an evaluation that works for everything. Btw, some of the parts I liked less are the ones written by Linus :-) Because he tends to make things a bit implicit in certain cases apparently. The VFS layer is an example.

I'm interpreting "finest codebase in [x, y category] that I've seen" to mean only what it says - out of the codebases he's seen, which may not be many, it's the finest.

I tend to run into new large codebases pretty frequently just from having to work with them. I'd say that happens maybe once every month or two for me, and I'm not even prioritizing this. I literally didn't know Ruby two weeks ago and now I've read through RSpec because I needed to submit a patch to some RSpec tests that were doing something unusual. For a CTO who's listing all the things he's seen his entire career, is two dozen codebases really that surprising?

I'll echo the sentiment. In our working life we don't get to know many codebases in depth. And the ones we do know, it's not at all obvious if they are relevant, or a regular disposable mess.

Start with looking at the unit test, it will help you split the code bases to units and understand them one by one