Graphviz 7 | Hacker News Reader

Graphviz 7

gitlab.com

205 pointsnikolay89 comments

I use Graphviz dot syntax to configure entire cloud computing environments in the order they must be built. This is pipelines as code and infrastructure as code

It's very powerful! Not every organisation can bring up an entire environment with one command of documentation their infrastructure with Graphviz.

Here is a executable diagram of a build worker cloud nodes, Kubernetes, consul, vault, Debian package server, Java app, SSH bastion, Prometheus, grafana, DNS and security groups.

https://github.com/samsquire/mazzle/blob/master/docs/archite...

The tool also parallelises the build based on the graph so for example packer builds can run in parallel.

https://devops-pipeline.com/

It also has a GUI but as a tool it's not ready for other people to use.

o wow! i know that graphana can be used with mermaid diagrams, but this is a whole new (lower) level. it would be nice if the graphviz could be styled a bit nices (like with sketchviz), but that's icing on the cake.

Holy macaroni, this is one of the coolest things I've seen in a while. I wonder if a top-level post of this could generate some interesting discussions about other people's exotic approaches to "DevOps", too?

Unfortunately I tried promoting my things too much so most of my submissions are blocked by the self promotion filter.

Yes I would love to hear other people's solutions to DevOps problems. I find it an interesting space. There's lots of good ideas but they haven't been brought together.

Very cool.

I created an analysis compiler that takes intent information and makes it part of the Enterprise DevSecOps pipeline in order to share realtime analysis with the entire organization. As I continued that work, I've also started talking about the feasibility of creating a supercompiler.

GraphViz is the shit, man. It was exactly what I needed to work on this material. You can create your own notation and then couple it with all kinds of tech by way of batch. Highly recommended.*

*The hardest part, frankly, was unlearning all of the other diagramming paradigms I've used. YMMV, but I found GraphViz most useful when you're going completely greenfield.

I love these words "As I continued that work, I've also started talking about the feasibility of creating a supercompiler." and thoughts already.

One of my ideas is a supercomputer compiler

I wrote about it here

https://github.com/samsquire/ideas4#112-compiler-as-a-servic...

Application performance monitoring, scheduling, control loops ala Kubernetes, parallelism, multithreading, data scheduling (shifting and sharing data to where it needs to be used), advanced caching and regeneration and container scheduling and binpacking and data layout scheduling could all be compiled by a supercompiler for extreme performance.

We are very much on the same page. I wrote about it here

https://danielbmarkham.com/outlines-of-a-supercompiler-in-f/

Our industry tends to over-complicate things and get in its own way of making the world a better place. SuperCompilers are a worthwhile effort to fix that. We need more forces for good. Kudos.

"an analysis compiler that takes intent information"

What does this mean?

1. Whenever anyone in the organization has an interesting conversation about the product they take notes. They're just plaintext notes using notepad or whatever.

2. There are about 20 keywords. They are represented as ALL CAPS

3. When you take notes, you tag them with the keywords

4. These notes are checked into git like everything else, and built as part of the dev/team/pmo/enterprise DevSecOps pipeline

5. Doing this creates a realtime and ever-changing analysis model

This leads to better conversations, better questions, elimination of duplication and conflicting business ideas, entry of data once, tracking bugs and features in plain language all of the stakeholders can understand, automated testing frameworks, interactive diagrams showing what the business thinks the world looks like and how they're products will change it, and more.

It was fun trying to design the simplest system possible able to map the almost endless complexity of human interactions around a given domain. tl;dr "Intent" is everything that happens between humans to organize their view of the world and the tech they're making to interact with it. Intent begins with brainstorming, perhaps BS sessions with coders, marketers, customers, and so forth and ends with feature tests, user manuals, and everything but the code itself.

I am big on state machines. Even simple tooling around them makes life easier for a lot of tasks. We often poorly implement a state machine in our code. Formalizing it in some way often makes life easier for me.

I’ve kind of considered something similar to this approach but ended up using a graphdb to keep dependencies instead.

Using a db gives you a better way of modifying meta data in batches which might be required at scale. It also makes it possible for tooling to query stuff quickly and do operations such as setting consul intentions, register something in an api gateway or other surrounding config tasks.

The pipelines we built checked out complete subgraphs from an api and generated json that referenced versioned TF modules and committed these to a service repo.

Thought about open-sourcing for a long time but never could get the go from legal. :(

This is very cool though!

I love GraphViz. Many are saying the layout isn't that great and there are better options, etc. etc. and I have no doubt that for some use cases, that might be true. However, GraphViz "just works" for me. Every time I build a tree-like structure, I simply open up a new .gv file and just start typing nodes connected with an arrow ("->"). I can do this without taking ANY extra time - I just type it in as I'm developing the structure. When I'm done, I have a very nice diagram. For the more complex ones I add some color or other small details, but often they are simple black and white unidirectional graphs. I keep them open while I work and they help me visually see my structures which helps me see the problem domain more accurately and write better code.

Shouldn't the extension be .dot?

.gz is already used by gzip[1]. This wiki[2] says it is .gv . I've always used .dot so I don't know what to use anymore.

edit: according to this[3] discussion , Microsoft uses .dot. I then found this wiki[4] page that supports the claim. dot is used by Microsoft Word document template files. I will switch to .gv

[1] https://en.wikipedia.org/wiki/Gzip

[2] https://en.wikipedia.org/wiki/DOT_(graph_description_languag...

[3] https://forum.graphviz.org/t/what-is-the-file-extension-for-...

[4] https://en.wikipedia.org/wiki/List_of_filename_extensions_(A... [4]

I've used .dot in the past and stumbled upon issues with git related tools (I believe git diff) interpreting the .dot file as a word related file. I'm using .gv now.

Oh right, back in the day .dot was a template file.

.doc = document, .dot = document template.

Typo: meant .gv (edited above)

Changelog: https://gitlab.com/graphviz/graphviz/-/blob/main/CHANGELOG.m...

No idea why v7 is trending now. The big changes (moving to C99) are in v3 from 2022-02-26.

That's a lot of major versions in just a few months!

Looks like they consider any breaking change a major rev bump. And it looks like most releases since then have had at least one breaking change.

Great tool and nice to see such active development on it!

From 3.0.0 to 7.0.0 just during this year, though.. ? I guess very few are depending on the library interfaces which I imagine are being broken, or do the major version increments just signify nothing of importance?-)

It seems to me that they bumped the major version for every breaking change in the changelog. I guess that is in strict compliance with semantic versioning but indeed it looks strange.

Wild, seeing SemVer actually used properly.

You don't need to do a new release for every checkin. You can make a series of breaking changes and only then declare a new release, bumping the version number.

After all, it took (at least) 21 years to get from 0 to 3.0.0

There's absolutely no reason for 2 major bumps 11 days apart.

If you’re doing CI/CD, though, you should expect this kind of thing. I do appreciate that maybe there could be a throttle for major bumps.

If you're doing CI/CD and pulling from head rather than official published versions, you deserve what you get. If you mean the graphviz project itself, version numbers are meaningless to it internally.

> You can make a series of breaking changes and only then declare a new release

That means you need to postpone the breaking changes for a while, kind of bunching them together. Many projects do that, but it also incurs some cost.

> After all, it took (at least) 21 years to get from 0 to 3.0.0

It's possible that they did not implement semantic changes in the strict sense before and included small breaking changes into non-major releases (I think a lot of software does this). Fast changes of version numbers may just mean they started reflecting each small BC break into the version number.

Right, we are witnessing a project switch to semantic versioning that did not have it before.

I don't know when semantic versioning was invented, but it only became prominent in the last decade. Graphviz is from the 90s.

Graphviz is from earlier than that -- mid to late '80s, I'd say. The first paper appeared in Software Practice and Experience in 1988.

The thing is, if you're writing software with a lot of breaking changes, SemVer doesn't make much sense, because you'll just quickly get to major version numbers in the hundreds.

Especially when the API surface area is as gigantic as graphviz's.

Still, it's great to see new maintainers with bold ideas.

> The thing is, if you're writing software with a lot of breaking changes, SemVer doesn't make much sense, because you'll just quickly get to major version numbers in the hundreds

“The version number conveys whether there is a breaking change” seems more important than “the version number remains small”, so while the second half may be true, I don’t see how that supports the first half at all.

The problem isn't that it's small, the problem is that if few changes are breaking, then you have "version 5" which is significantly different than "version 4".

Is version 465 very different from version 464 though?

/me points at Firefox

A bit unfair, no? Firefox switched to the "new major version every 6 weeks (or so)" only because Google did that first with Chrome, and people started thinking "Chrome is so much better because it's in version 17 and Firefox is version 3".

> Is version 465 very different from version 464 though?

As soon as: a) you precisely define "very different" AND b) versions 464 and 465 of Firefox and Chrome are available, we can continue discussing :)

I made a DOT language module for BBEdit if anyone is interested https://github.com/VadimMuhtarov/DOT-BBEdit-CLM

I liked GraphViz a lot, but eventually got frustrated with its layout algorithms and default look and feel. I would switch to Mermaid wholesale if it could render SVGs without requiring a browser engine.

I’ve personally settled on grapviz (a derivative actually, plantuml) being “good enough” for casual diagrams. Plantuml is built with the idea that it’s for quick and dirty communication, and I think graphviz translates well to this. It’s a nice way to put together a sequence diagram that just helps you understand control flow (and then never took at it again).

I have to admit, I haven’t found a more formal programmatic tool. Ultimately I have to turn to “by hand” tools if I want something more complicated. If you can’t translate that last statement, it means I just never do anything more complicated.

The next step are tools like featdoc which let you define systems and interaction rules which then generate a (mermaid) sequence diagram.

Example at https://dev.azure.com/itmv/Coffee%20Shop/_wiki/wikis/Coffee-...

Project at https://github.com/Calpano/featdoc

Disclosure: I maintain this project

For mermaid, would pairing it with selenium and/or a headless browser be an option?

Not sure if this question is directed at me. My understanding is that mermaid is relatively similar to plantuml in that it doesn't give you 100% control over layout and diagram features.

What plantuml lacks (maybe mermaid less so) could really only be improved upon by working with raw svg I think. There is a LaTex package tikz that offer much more flexibility than plantuml/mermaid, but you still run into limitations in that diagram features are predefined.

Ah thanks for your answer comfypotato, Tikz looks even better then Graphviz. No, I misplaced my remark in the thread and directed at rcarmo just one level above who would have switched to mermaid if it didn't need a browser (or perhaps just node.js). Tikz also needs a LaTex dependency, with gigabytes of diskpace I'm personally not that frugal anymore with tooling size even though I like using as little disk space as possible. Perhaps rcarmo reads the thread further, perhaps not.

Heck no. Why can't it generate the SVG without a browser?

I also promote PlantUML for projects, specifically because:

1. It provides a convenient DSL targeting specific diagram use-cases that I actually want (generally sequence diagrams.)

2. The "source code" for the diagram is trivial to check in and diff with version-control, sometimes right along with the code it describes.

3. If the app/renderer is for some reason not available, the source is clear enough that you aren't completely lost.

I hadn't heard of Mermaid before this conversation, but it seems to hit some of the same notes.

As I said before, I'm a big fan. I do wish there was a sister project that was intended for a higher level of detail. PlantUML has explicitly stated in some forum responses that it intentionally limits your ability to place elements exactly where you want them. I totally understand their decision, and I greatly enjoy using it for its intended use cases. There just isn't really anything that comes close to the practicality of PlantUML for more detailed diagrams, and I feel like PlantUML would be the best starting place for such a tool.

The creator of PlantUML is working on translating graphviz into a native form to eliminate the dependency for PlantUML use. Once this has been done, it would open up even a whole new world of possibilities (graphviz does allow a bit more specification of element locations).

Sketchviz looks a bit nicer, but also 'sketchy' https://sketchviz.com/new Regarding mermaid, see my misplaced comment one level deeper. I'd think that with node.js it can export to svg.

Last I checked it actually used puppeteer to launch a Chrome instance to render files, which was ludicrous.

Graphviz is excellent. I use it for (amongst other things) generating diagrams of states and transitions, and also for automatically mapping choices as I write interactive fiction.

More generally I've added it to Newt [1], which is my in-progress C# code generator. It scans a PostgreSQL database to automatically generate a class library with models and EF Core data contexts.

As part of that it writes out a dot notation Graphviz source file for showing the tables and relationships. The repo includes an example image [2] - it only shows a couple of tables, but it gives an idea of what can be produced (and also shows the .dot source file that was created to produce it).

[1] https://github.com/kcartlidge/Newt

[2] https://github.com/kcartlidge/Newt#created-graphviz

Edit: If anyone does C and has skills or money available (I don't) the Graphviz README says "This project has only 3 of its founders remaining as volunteer maintainers, and support is limited and may end soon. If you can help fund to a more formal foundation to continue this project, please contact north on graphviz org."

This is more for diagramming small graphs. If you are trying to render large networks, there are better alternatives.

For OSS graph visualization libraries, check out: - Sigma.js - Vis.js

For commercial options: - Ogma (disclosure: my company builds that one) - KeyLines

You can’t mention commercial options without mentioning yWorks. Their product is absolutely amazing and I’d use it for everything if it wasn’t for the steep cost.

(Big fan of graphviz, it was such a shame when AT&T pulled funding!)

Maybe helpful, more for visual graph data analysis than say arch diagramming: we build Graphistry specifically for interactively analyzing more data than these tools, eg, GPU client + GPU server, and being more batteries-included. The free Graphistry Hub site is similar to diagrams.net and GitHub for folks who aren't self-hosting GPUs.

We are still working towards OSS'ing the core renderer, but all our new graph neural networks + graph compute + graph layout layers are all already OSS-first (see PyGraphistry), and we are part of the apache arrow + rapids communities. If your kind of thing, we are very much hiring :)

As someone working at yWorks on yFiles, thanks for the kind words :-)

Having used Graphviz extensively over the years as well, the cost of yFiles is certainly unfortunate, but I guess we simply serve a different markt segment than Graphviz or D3. (On a somewhat related note: Our online graph editor, yEd Live, support parsing a subset of Graphviz' syntax and that enables running at least some yFiles layout algorithms on Graphviz graphs. No export back, though.)

Are Sigma and Vis widely-usable (i.e. have C bindings), or only for browsers/JS enginers?

+1 for Vis.js

I've used it to map wireless trading networks with lots of connections.

The fact that you can make the nodes clickable to other webpages etc is a great feature for building "drill down" views.

js.cytoscape.org is pretty cool too for fast layout of large nets

One of the better graph visualization tools!

Which tool do you people use most often for graph visualization? I kind of linger between networkx, Gephi, graphviz and d3.

I really struggle with this. I also use networkx, Gephi, and graphviz, and occasionally plotly. D3 is too much of a learning overhead for me at the moment.

I'm really in the market for a decent engine that's both easy to use and enables high quality interactive exploration of massive datasets. Plotly comes close, but struggles with larger datasets and is still a bit fiddly. I'd love to be able to lob a large structure into a UI and share with colleagues, and they'd be able to visualise and filter based on node/edge metadata, time series etc.

Something like Gephi, but web-based and considerably easier to use.

Agree. Neo4j’s visualization tools are surprisingly painful to use. I’d love to find something that allowed you to trivially add code to the layout engine. Graphviz’s clustering capability seems to be unique among any of the tools I’ve used as well.

Gephi can cluster, but the algorithm isn't that transparent or flexible. I've increasingly found myself doing the clustering in Python and rendering in Gephi.

That's why we built Graphistry :)

Can be used code-free (drop in a csv or log file) or via analysis workflows like from Jupyter notebooks and internal dashboards.

We are roughly parity for Gephi's features, and pushing (far) ahead on web, GPU, perf, ai, automation, etc. Always curious about gaps - our Slack is a good place to chat on that.

Heh, HN is a great place to fish. This looks great. I'll have a play around with the free account. Thanks for sharing.

I often use https://github.com/Rubjerg/Graphviz.NetWrapper to export the graphviz layout as json and then use d3 for rendering :)

Someone needs to make a multi-OS clone of omniGraffle. I paid €179 for omniGraffle, I’d pay for a multi-OS alternative.

Does omniGraffle accept text input?

Because one of the points of GraphViz is text input, which can be easily source-controlled, diffed, machine-generated, etc.

Yes Omnigraffle accepted Graphviz files. But uses its own excellent layout engine and allows you to adjust the layout by hand. If you include integration with OmniOutliner, you also get Graffle can also use a wide variety of text based outline file formats.

https://support.omnigroup.com/doc-assets//OmniGraffle-Mac/Om...

Amusingly, graphviz had dotty way back in the day so that you could adjust by hand. Now, it looks like they have gvedit. I haven't tried that one.

That said, the ability to "fix" the location of some items by providing it in the dot file is a feature most folks seem to ignore. Or not know exists.

I objected to how often I had to upgrade the version and pay for very minor fixes.

I bought it 5 years ago and haven't given them a dime since, and got a lot of new feature since (scripting!). I'd love to have more library, but that's another issue :)

Draw.io

Not really. Draw.io is browser only. I like my diagramming tool offline.

There is an offline version: https://www.diagrams.net (they had to change their name from draw.io recently). There is a download button right on the main page.

That's awesome. Thank you.

Which looks to be an Electron app, so there's a browser engine doing the rendering. Can I save a diagrams.net/drawio-desktop drawing into a file, and convert that file into an SVG in a lightweight batch job?

Yes

Really handy tool

Probably saved me a ton of time that I would spend on implementing visualization myself

same here

I use graphviz via networkx to build a dependency tree for my company's app. I'm able to split one master tree into sub-trees based on which "flow" I want to look at.

img: https://imgur.com/7qCKA1G code: https://github.com/hammeiam/saddle-data-graph/blob/master/Sa...

Out of curiosity, has anyone settled on a method for storing and surfacing metadata about a node?

Example: a server has an IP address, dns entries, a description, and some other fields, and it connects to other servers. Is all of that meta information in a JSON object in the key-value area of the node in the .dot file?

Does anyone know of any DOT -> mermaidjs format converters? I was looking for one recently and was surprised not to find anything more robust than someone's quick search and replace script.

Does anyone want to look at porting the mermaid layout method back to Graphviz? It might be useful to people. I guess some friction would be around the kind of awkward way that Graphviz has to deal with objects that aren't strictly nodes, edges, or subgraphs.

I used graphviz to process the output of clang format, to visualize function calls between functions.

The output was quite messy.

I don't really know for to give clues to it, I tried several options.

Graphviz is one of those things I wish I had found much much earlier in my career. It makes throwing together a little one off tools to chart data relationships effortless.

https://diagrams.mingrammer.com is great - requires graphviz

From the readme:

"This project has only 3 of its founders remaining as volunteer maintainers, and support is limited and may end soon. If you can help fund to a more formal foundation to continue this project, please contact north on graphviz org."

You can ask us anything here.

The community should applaud the efforts of Magnus Jacobsson, Mark Hansen, Matthew Fernandez, Steve Roush, and recently Costa Shulyupin to support graphviz in a more disciplined and professional way. In hindsight, we got a lot of play by treating Graphviz as a research vehicle, but we weren't able to (or didn't choose to) direct a lot of effort into documentation, rewriting existing code, sorting out feature interaction, developing very polished front ends, or broadening the scope of the effort. Or finding a new home after AT&T Labs lost its enthusiasm.

I am glad to see that they recently (relative to project age) got two new developers which have for the last two years actively worked on rejuvenating the project, tackling some long standing issues, and actually making new releases with bugfixes. That message was added to README before that. Of course it doesn't mean people should stop supporting the project, but I am currently less worried about future of graphviz than the last time I looked into the state of it.

Reply on news.ycombinator.com