Tools for visualizing a codebase | Hacker News Reader

Tools for visualizing a codebase

lmy.medium.com

148 pointstslmy37 comments

Most of the tools for visualizing code lack the practical part. Dependencies graphs are often so huge, that generated graphiz output doesn't even open on Macbook M1. Codeflow graphs kinda helpful sometimes, but nothing game changing.

I explored the topic of visual programming a couple of years ago and presented the first attempt to make the code visualization tool useful and practical in this long-read - "Rethinking visual programming with Go" [1]

The key point is that we build mental maps of the code whiel reading it. Simplicity of Go allows for rules' formalization of how I build 3D mental map myself, so I can outsorce it to the code. I.e. the static code analysis tool that reads any Go codebase and visuzlies a 3D structure similar to how I would do it in my head anyways. By clearly separating spatial and temporal aspects of code, it's possible to blend this code visualization approach with editor and make it practical and helpful rather than just visualization excercies.

I paused the development, as I see the future of this project only in VR/AR environment, and I'm not sure this ecosystem is ready yet (it's a side project still). But honestly I dream about this tool almost daily, while coding with usual text code representation.

[1] https://divan.dev/posts/visual_programming_go/

Amazing write up. Spent 25 minutes on it.

I’m about to start learning Go now. I tried it a few times before but I never got that ah-ha moment. I wanted to build a visual tool in the devops space, and go is prevalent so I need to learn it sooner or later.

But with your explanation, I see similarities to OCaml (which I love), where most of the work is modeling the domain by composing types.

> Simplicity of Go allows for rules' formalization of how I build 3D mental map myself, so I can outsorce it to the code.

I don't see how that's any different from reading Java/C#/Kotlin code honestly.

Very interesting article. I'm gonna read it in more detail tomorrow. Thanks for sharing it.

However, I did note that you lumped LabVIEW in with other visual "tools" and scripting tools. LabVIEW is a fully featured, general purpose language. You can do anything in it you can do in other languages, and there's plenty you can do in it that other languages can't. I don't know of any language that allows you to program desktop PCs, embedded high-level programs, real-time programs, and FPGAs with the same language. Maybe C/C++ with Vivado HSL. But...C/C++.

LabVIEW compiles to DFIR, an intermediate dataflow language, and then that compiles down to LLVM IR, which is handed off to LLVM. You can call .NET assemblies and C/C++ DLLs. It has value-based OOP, various (rather unique) implementations of polymorphism, built-in multithreading across cores, immutable data, a nice event structure for building GUIs, is highly performant, and a ton of other features. It has built-in support for basically any protocol (HTTP, TCP/IP, UDP, etc.), so it can talk to basically anything. If it doesn't have something built-in, you can obviously build your own or pull in some .NET or C/C++ dependency. So yes, you could rather easily write a raytracing engine or a GraphQL library in it. In fact, LabVIEW is damn fast, so it is great at things needing to be performant, especially if they can be parallelized.

LabVIEW is "specialized" to certain industries for three main reasons. (1) How well it ties in to NI's hardware and other test and measurement and scientific hardware. (2) It is expensive, but now there is a free Community version. (3) Non-acceptance amongst "mainstream" programmers of visual programming languages. So it actually has little to do with the language and more to do with factors external to the language.

Inside every large graph is a small graph dying to get out.

There's a place for just wanting to see everything, but the beauty of graphs is abstraction. Sometimes if you are asking a specific questions, a small, beautiful graph might be a better way to answer the question, than just rendering a diagram with hundreds or thousands of objects.

Graphviz just provides a basic rendering service; it would be up to higher semantic layers to support such exploration. Also, no doubt, interactivity is essential. It would be good for some enterprise that has the incentives to support the work, to create a new system, or at least to extend Graphviz to support better interactivity. It is not such a big stretch.

Most visualization tools (not just for codebases) seem to rely on graphviz. Is there anything better? Or is this still the best option?

Yup. Seems like graphviz is the de facto standard.

Better in terms of? Style? Performance?

Maybe something interactive with d3?

Trying to visualize your codebase with a format that is even more static and less navigable (a static image) feels a little contradictory imo

Gource

https://gource.io

Back in the day, Visual Age for Java (IBM) had a cool dynamic call graph visualization tool. Would be great if companies released their tool sources when they abandon them.

They also had Visual Age C++! AND it ran on OS/2!

For a more dynamic moldable approach, look at Moose. https://moosetechnology.org/

CMake can also generate a graph of the targets with graphviz, which can be very useful (and also make horrible things like cyclic linkages evident).

Back in the ancient times of the early 2000's I had to deal with a massive, legacy java codebade.

I found a small free library called MaintainJ that used aspect-oriented programming as you used the app. It was great to make sense of how everything was pieced together: You coul fire it up, run through a single scenario (user log in, add to cart...) and generate a flow diagram of just that scenario.

The library still exists, http://www.maintainj.com but I haven't used it in +15 years, so no idea how it is today.

This is close to my heart :) I've been toying with a codebase visualizer, "Shotglass". It's unique that it shows a pixel _for every single line of code_, thus showing the overall structure. Functions can be individually colored, e.g. to show code complexity.

Code + screenshots: https://github.com/johntellsall/shotglass#demo-django-a-bit-...

http://epydoc.sourceforge.net/api/ - epydoc has great combination of reference documentation combined with codebase visualization. Please note that the two visualisations in "Package epydoc" and "Package Organization" ar both clickable. Once clicked, also feel free to find and click "call graph", which itself visualizes, well, call graph.

Sadly epydoc is no longer maintened and only supports py2. Had an idea of porting it to py3, but never found enough time to do so.

A couple of my own projects might also be of interest. Repography [1] creates posters with more of an aesthetic focus, and Work/Artwork [2] takes this even further, creating modern art.

[1] https://repography.com/app/0/neovim/neovim/trunk/poster-surf...

[2] https://workartwork.org/design/mountain

Gource ( https://gource.io/ ) is awesome if you want to visualize the evolution of a codebase fyi.

+1 on gource. Very customizable, can be used to render video, shows contributors as little flying people, and it does offer some interesting insights depending on what you're after.

I feel like there should be an IDE tool that shows a dependency graph and you can click on each function to go to the code.

Functions happen to live in the same file, but they can really be their own thing, especially pure functions. Even just having a tool to pick out pure functions vs state modifying functions would be great.

I want a graphical way to focus on each building block in a codebase that moves beyond the file format.

https://www.jetbrains.com/help/resharper/Architecture__Proje...

Ver helpful post. This is an under-covered topic especially for large code bases.

Anyone aware of efforts to pair these tools with tools to filter and examine different call paths to interactively explore a large code bases. A model like specifying a filter and having the view be filtered by paths that meet criteria.

Dependency graph visualization for Go:

    godepgraph -s package | dot -T png -o depgraph.png

With: https://github.com/kisielk/godepgraph

I've been meaning to give this a try: https://github.blog/2022-02-14-include-diagrams-markdown-fil...

I constantly use mermaid for flow charts etc in markdown.

Gitlab for example renders these if you tag a codeblock with "mermaid".

For JS projects, I use the script below to analyze dependencies, such as unused files, or files that are used only once and could be consolidated.

https://github.com/uxtely/ops-utils/blob/main/find-unused-de...

For monorepos, I check which files are shared between projects with:

https://github.com/uxtely/ops-utils/blob/main/show-interproj...

A dependency structure matrix is a nice tool to have for some languages too (used to worked fine from IntelliJ for Java at least 10 years ago and I bet it still does).

You should have a QuickStart. Use the ReadMe to explain what the QuickStart does. Then, use the QuickStart as a skeleton on which to elaborate things not covered.

Good idea! I should learn about the conventional sections in repo documentations.

Anyone know of some tools that could draw graphs of the Linux kernel ? I’d like to draw subsystems or pick a function and have it diffuse out to build a graph.

It's missing a very popular option:

https://sourcegraph.com/search

I can't see from their video what this gives over jetbrains find all? Except for a web interface that takes me out of the editor.

I'd say larger code bases spanning multiple repositories.

Anything for .NET?

You may want to look at NDepend[1]. I used it recently for a project and it worked pretty well but a full license is quite costly so I only played with the trial version.

1: https://www.ndepend.com/

Try Cidescene.io

Did you mean codescene.io?

Reply on news.ycombinator.com