Codegraph – static code analyzator / code diagramer | Hacker News Reader

Codegraph – static code analyzator / code diagramer

github.com

97 pointssargstuff26 comments

Nice work!

Other posts in this thread are touching on it, but graph layout and layer segmentations are where you could improve.

Screen real-estate is the biggest challenge when visualizing large amounts of code.

Similar to maps show different detail levels based on aspect ratio and zoom, you can decompose your "features" (mapping terminology) based on level, i.e. packages->modules->functions->code

OP, have you ever used Gephi? I used to build static analysis tools for viz of large-scale codebases. Taking a bit of time to export analysis data into a Gephi format (GEXF in my case) really helped.

In Gephi, you can play around with different layout algorithms (peep the plugins). This may help you select a layout algo or combination thereof to help render tons of data in a human-friendly form.

OP here's a couple examples of large scale static analysis diagrams.

[1] is a TypeScript database with ~250K LoC. A fairly advanced product that would benefit from what you're building. The colors represent different architectural layers. Each node is a folder with files in it. Edges represent inbound/outbound dependencies. There's simply no way for it to be labelled effectively.

[2] is the same codebase, but instead of inter-module dependencies, it's showing function/method calls.

One takeaway after working in this space for a while was that visualizing spatial relationships of anything sufficiently complex is extremely hard on a monitor. These representations benefit tremendously from being interactive.

[1] https://i.imgur.com/IdY5ghC.jpg [2] https://i.imgur.com/rv4OYun.jpg

Very nice. For those interested, you can get similar information using SourceGraph and LSIF in a standardized, language agnostic form: https://lsif.dev/. It still generally requires build information for each project/language, unfortunately.

Good concept. visualization is poor. Force Atlas is really not that great for structural analysis, and nodes need to be weighted by degree among other more code-specific factors. Consider bringing the code graph into NetworkX or some similar library and using graphML to leverage existing visualization solutions rather than reinventing the wheel.

I haven't looked very hard, but `pip install codegraph` didn't result in a codegraph command anywhere. (This was macos, using a Python 3.8 venv.)

Digging around a bit found a 'cg' command, which seems to be codegraph (based on the --help output). Didn't show anything interesting for a Django app, though... I'm guessing it couldn't really follow what manage.py was doing?

Anyhow, if 'cg' is the command, might want to update the docs.

README for codequery[0]outlines an overview of tool set to go from source code to visualization. (although, not as a generate live updates per editor changes)

[0] : https://github.com/ruben2020/codequery

Can anyone recommend something similar for Java / Smali ? Specific use case is reverse engineering Android application with partially obsufcated code.

Current approach is following “xrefs” in JADX and hooking methods with dynamic instrumentation with Frida (displaying backtraces etc). Got to be a quicker way while chipping away renaming classes and methods and mapping out source/sinks.

I built https://packagemap.co which is similar and designed for java. I'd love to get your feedback on it.

Sure! Will have a play and will reach out if appropriate.

Shortly before this was posted, I asked for a similar thing! https://news.ycombinator.com/item?id=35603022

Although in my case it'd ideally be language-agnostic and hook into LSP so that I can click on nodes and jump to various files/modules/functions.

Still a cool project though!

Radare2

https://www.radare.org/r/

emacs. LSP (under editor such as emacs[0]) provides the context/way for being able to define a specific language parse specifics about where to add/define emacs hooks/jump points for a given language nodes/files/modules/functions)

[0] : https://www.masteringemacs.org/article/tree-sitter-complicat...

I honestly don't wish to do the original poster down, but has anyone actually found code visualisation things like this (graph or otherwise) useful, as opposed to just pretty?

It seems like you would only really have need for visualisation if your codebase is complex enough. And if it is complex enough, visualising it is generally something of a mess.

What actionable insights can be generated? Has anyone changed anything as a result of such a visualisation?

It always feels to me like you're better off figuring out which modules you have shouldn't depend on certain other modules and finding only those links. Which is often best done with a simple ripgrep, no?

Unrelated, but curious: when "analyzator" should be used instead of "analyzer"?

Never. "Analyzator" is not a word.

The HN title matches the repo's README, which was authored by someone whose English language skills are questionable.

Python only?

Cool tool nonetheless, have built similar things to visualize code architecture and create doc artifacts in CI, usually the biggest challenge is to scope the pieces that will be included and what layout algo to use, maybe a feature to implement?

Emerge-viz seems to have multiple languages and cool visuals.

https://github.com/glato/emerge

Although it's discontinued, SourceTrail is still great for this purpose.

I was really surprised to saw a lot of stars on one of my oldest repos )) it was just for my purposes and I didn't continue work on it because thought that it is not needed at all ))

I'd be curious if someone loaded these into http://github.com/graphistry/pygraphistry and tried something like overlaying a GNN anomaly detection over the base visuals. Both parts are automated & GPU accelerated, so can probably handle some decently sized projects!

Hey, cool idea.

I built something similar for working with Java code, but it could be extended to other languages.

https://packagemap.co

I've seen engineers find it hard to work out if their code is organised well together. So I built this tool to show them visually.

That graph is illegible. I would remove most of the labels by default and show them on hover.

Agree but maybe only show labels for the largest modules and functions and also scale node size relative to function and module size.

Even doing that won't help.

Like, a reasonably complex program is going to have a call graph with tens or even hundreds of thousands of nodes. Graphviz won't save you. Collapsing sccs won't save you. It won't actually be useful to navigate.

They could collapse nodes into clusters, and reveal them on zoom, etc.

This needs more sophistication to be useful is what I'm saying.

Nice work !! Quick question: does this read python2 code or only 3?

Reply on news.ycombinator.com