Graphviz in the browser | Hacker News Reader

Graphviz in the browser

mdaines.github.io

323 pointsTAForObvReasons69 comments

Open articleView on HN

My Graphviz in browser with embedable alien URL.

https://graphviz.gomix.me/

Source: https://gomix.com/#!/project/graphviz

Sample graph in SVG:

https://graphviz.gomix.me/graphviz?layout=dot&format=svg&mod...

Technically, that's GraphViz as a service, as it's actually executing on the server.

This is executing clientside (for better or worse; yours is much faster :) )

That's much nicer than mine at http://fiane.mooo.com:8080/graphviz/ , I was using another online graphviz service a while ago, that suddenly went offline - my google foo didn't find any other that worked - so I decided it was a nice starting project for learning some web development and Go.

Nice to see Graphviz trending.

I created http://agile.pics a couple years ago based on the .js graphviz library.

The site is a thin wrapper that renders Graphviz of Sequence Diagrams from GitHub gists.

Example: http://www.agile.pics/gist/163647fe40ae4911e469

> Nice to see Graphviz trending

agree completely; GraphViz is extraordinary.

for intance, Yifan hu (U Florida faculty, i think) maintains a spectacular gallery of very large graphs comprised of millions of nodes, all rendered using GraphViz (persisted and input to GraphViz as incidence matrices): http://yifanhu.net/GALLERY/GRAPHS/index2.html

The graphs in the gallery i liked to above were rendered using sfdp, one of the 5-6 layout engines in GraphViz, and which is the GraphViz engine of choice for very large undirected graphs not limited to a particular layout geometry

This is a beautiful thing. D3 can do a lot but very few programs have replicated what graphviz's dot can do.

There used to be several commercial alternatives in this space (browser graph drawing): jgraph/mxgraph, tom sawyer, yworks (yed) had a browser library. It's curious that these never get discussed when this topic comes up. It's been a few years, so they may no longer around. The hard part of graph visualization has always been the layout algorithms. I tend to think that the ubiquity of graphviz has been an impediment in this area (imo)

Considering I'm still getting paid, I can say that yWorks is still around ;-) Since about a month with a new major version of our JS library as well. mxGraph apparently transitioned to open-source with paid subscription by now – we'll see how well that works out.

I guess the reason that commercial solutions rarely get discussed in such contexts is that they are quite expensive if you're a single developer hacking away at a side project. Even our cheapest license is already five figures (considering Tom Sawyer also rather target enterprise customers they won't be cheaper) and to be honest, I'd think hard about paying that if I'm not absolutely sure I'd get the money back somehow. For larger companies it's less of an issue, of course, or the cost just gets billed to their customer as part of the project cost.

Ah I see, since you're on the inside I'll ask, why haven't you (i.e. yworks) implemented the so-called delta drawing for flow charts? Seems like that would be a real differentiator? Or have you and I haven't been paying attention?

I have to admit, I'm not familiar with that¹. Do you have a link to a paper or something; I could ask our layout team.

But probably the answer will be that few customers asked so far and/or it might be fairly easy to accomplish with customizing one of the existing layout algorithms. We did something similar with our example BPMN implementation where a pre-processing step applied constraints for the hierarchic layout based on the node/edge types in the diagram. The vast majority of things customers ask of us are actually fairly easy to implement without necessarily having them in the library. So we have very few layout algorithms that are applicable to exactly one kind of diagram (family trees are an exception, but even that is just a wrapper around a more general-purpose algorithm).

The advantage is that there are very many knobs and configuration options to fiddle with to get the layout you want. The disadvantage is that there are very many knobs and configuration options to fiddle with, where having special layouts for certain types of graphs might be easier but not as flexible.

In the end, it's a library and the users (from our perspective) are developers and those shouldn't shy away from writing some code. It would be nice to also appeal to people who just want some data laid out nicely. To some extent that works already², but such wishes have to be balanced with what people are willing to pay for :-)

______

¹ Happens often with very domain-specific graphs; last thing mentioned I didn't know was some kind of diagram to describe pipes, valves, and water flows in buildings.

² e.g. http://live.yworks.com/yfiles-for-html/2.0/databinding/graph... has only very minimal user code

Your ¹ sounds like the Sankey diagram[1]. I only know about these because at one time matplotlib advertised support. And because of the classic diagram that Tufte promotes [2].

[1] https://en.wikipedia.org/wiki/Sankey_diagram

[2] https://en.wikipedia.org/wiki/Charles_Joseph_Minard#Work

I still think dot could be reimplemented in d3. I agree dot is very expressive but it's starting to feel like TeX where it works great until you need to mix it into more recent applications.

dagre.js uses DOT. https://github.com/cpettitt/dagre-d3

it uses the dot language, but not graphviz's dot layout engine

Phabricator had allowed for specifying dot notation in their markdown (remarkup) which would use graphviz to render the images in the comment. I loved this feature but it was removed due to potential exploits [0]. Is anyone familiar with how graphiviz-as-a-service could be possible while addressing these concerns?

[0] https://hackerone.com/reports/88395, https://secure.phabricator.com/T9408

As both a heavy phabricator user AND a graphviz lover, your comment got me quite excited....up until the part where it was removed :(

If you look through the task description on the phabricator task I linked, there is Paste file which has the contents of an extension to allow continued use of graphviz, however they state it's insecure and not supported.

My guess is that seccomp blacklisting file access should be enough. It wouldn't be too hard to patch graphviz to disallow file access. And/or containerising it to only have write/read access inside a single, dedicated folder.

Graphviz has code to restrict file access when used in a web server. Its in lib/common/utils.c:safefile()

But to be honest, a container approach would probably be better today.

This is built using Emscripten. They compiled native Graphviz to JS and there is a thin shim that renders the .dot files to the browser. Dot command gives output that makes this relatively easy.

This is very cool. I love graphs, they're really expressive.

P.S. if you're interested, just the other day I posted a related experiment of mine on Show HN ... https://news.ycombinator.com/item?id=13308150

This may be a tangent, but I feel like pointing out that I, for one, have recently become more aware of the fact that a graph is really just a binary relation.

This is not discussed much in all the talk about social networks. It's always the graph (the "one") that's foregrounded, rather than the multiplicity of edges, each a relation.

I know this is mathematically an obvious thing, but surely the relational perspective is more amenable to generalization. We should make more of it.

Mostly, if you cant have multiple edges, that is true.

If you can't represent some well in a graph because it's too messy, you could try using an adjacency matrix, along with a clustering algorithm to figure out how to order the columns / rows.

Actually, if you consider each edge as an entity, you can represent multiple edges using the relation incident(Node, Edge). This is analogous to using an incidence matrix. It also works for hypergraphs.

If you're thinking of weighted, directed graphs, a binary relation doesn't quite fully represent it. In the weighted directed case, a graph is a function from Nodes × Nodes → Weights, (where Weights ⊆ ℤ or Weights ⊆ ℝ or something like that)...

Which makes weighted graphs perfect for representation by matrices!

Studying adjacency matrix of weighted, directed graphs gave me a profound realization matrices are a table of relationships between their "dimensions". I never looked at matrices the same way again. I realized that the identity column was literally the entries that represented a relationship between a dimension/node and itself. I went down many rabbit holes... particularly the question of graph isomorphism. That problem is a rabbit hole. Another rabbit hole is the Hamiltonian cycle problem.

P.S. check out hypergraphs, which are not necessarily binary relations and generalize graphs. Incidentally, hypergraphs have an adjacency tensor. I've started to understand that tensors are like 'tables' of relationships between dimensions + dimensions representing some or all of their possible combinations.

By the way, if you can figure out a way to do fast multiplication of matrices over degree-truncated polynomial rings, I'll show you a fast way to count the number of hamiltonian cycles in a graph. These problems are intricately linked.

Sometimes the matrix, like for example the adjacency matrix for Wikipedia internal links, will be very sparse. Another reason to use relations rather than an inefficient, mostly empty implicit data structure.

That Wikipedia adjacency matrix would be interesting to visualize, though. There are tools out there for visualizing sparse matrices as graphs:

http://yifanhu.net/GALLERY/GRAPHS/

"using relations" or a particular data structure wasn't at all the point of this particular discussion.

The point was that, for a simple directed graph, the edges represent a binary relationship. While for a weighted directed graph, it represents a relationship-matrix. For the graph being non-directed this would mean, that the relationship(-matrix) is symmetric.

Meaning that any algebra, algorithm, proof, etc. on graphs of that type can be applied for the other interpretation as well.

Implementation is a different question. "Most" binary relations we happen upon (less-than as the obvious) have an infinite domain and infinite cardinality (when represented as a set of tuples), making it very hard to "visualize" as a graph.

Your last sentence is pretty much a summary of my attempt to understand the amplituhedron, and as far as I've gottan

I've never seen a visualization of a graph at the scale of Wikipedia that conveys anything useful in the visualization. You just get a big hairball.

Yeah, actually that page has a Wikipedia graph on it which looks exactly like a spiky sun.

I'd like to see more work like the paper from a few years ago which ran PageRank and HITS on Wikipedia to discover the most central pages. The PageRank result indicated church hierarchy and nation states were important, while HITS had things like "television" "animal" as the most authoritative/central pages.

There is so much structure there to be investigated.

The best you can usually do is clustering, but at that point you can just as well render clusters as single nodes and significantly lessen the strain on the visualization or the layout. I'm always astonished when customers ask about visualizing graphs with tens of thousands to millions of nodes. Thy usually have never thought about the UX implications of that (let alone the time needed to calculate a layout which results in an image so large that each node is smaller than a pixel anyway).

This is a very good point. In the early days of Graphviz we put a lot of effort into making very readable diagrams of fairly small graphs, mainly in drawing directed graphs (dot). We aimed at graphs with under a hundred objects. We agree with the viewpoint that it is better to apply some sort of analysis to reduce a larger graph to a smaller one that can be read and understood. People still want to draw much larger networks. In fact it is valid to want an overview of all the data, it's just not clear how to do this with many thousands of random connections. Graphviz sfdp (based on Yifan Hu's earlier work) and recent work on Maxent (which has a decent theoretical justification) and from Ulrik Brandes group are good examples. One reason graphviz neato does not make such great layouts of large graphs out of the box is that it is straight statistical multidimensional scaling, and we just drop shapes on top of the points, so they can overlap badly. We felt since people are relying on it for data visualization we did not want to post process the MDS layouts by default in a way that could be misleading. But you can turn on overlap removal (neato or sfdp -Goverlap=false). We really should document all this in one place that is easier to find and understand.

About the external shape loader issue - this code is somewhat centralized in graphviz/lib/gvc/gvusershape.c and in gvrender.c which calls it, and I thought it could be disabled at compile time (because we did address the security concerns at one point) and there's a lot of other machinery to control compile time features. Maybe John Ellson can comment here. Kudos to John for recognizing the problems with the shape loader as soon as I proposed it but apparently that didn't stop us at the time. Stephen North

Here It Is a package that implements some hypergraphs features. http://g14n.info/iper/

I see plenty of libraries and tools for displaying a graph that has been defined in text code. Is there a really easy tool for generating graphs with a gui, easily and intuitively, and converting that into text? The solutions I've found don't make the node and edge creation easy enough.

I like using yEd[0] for quickly drawing workflow graphs. It exports as GraphML, but you'll have to convert it to dot yourself.

[0]: https://www.yworks.com/products/yed

Seems much like this: http://cpettitt.github.io/project/dagre-d3/latest/demo/inter... what are the differences?

So far it seems like dagre-d3 is the best option for those that want directed graphs to be automatically redrawn (with transitions) based off of user input. I'm continually looking for better options, though.

viz.js is Graphviz recompiled to javascript with Emscripten. Dagre is the dot algorithm implemented in JS. I found dagre to work really well in most cases but Graphviz beats dagre in some edge cases (e.g. not order dependent, better with hierarchies, ...).

Dagre / dagre-d3 is not actively maintained anymore, but I'm still using it in production and loving it. Very easy to use custom HTML and CSS, which is much harder with GraphViz.

Nice work!

I needed to use something to render directed graph in the browser and the biggest problem was the download size.

This library is 851KB gzip, which is way better than the 1.3MB I was using.

Maybe being able to bundle the engines (and output format) as separate modules could reduce its size?

So once again, great work!

I just implemented directed graphs with cytoscape.js and it was fairly straightforward.

WebAssembly would help reduce the size.

Mermaid.js can render directed graphs in the browser, and the minimised JS and CSS are ~620 KB.

https://github.com/knsv/mermaid/tree/master/dist

e.g. live online editor:

http://knsv.github.io/mermaid/live_editor/#/edit/Z3JhcGggVEQ...

If you just want to render something and don't need layout, this could be very small. I guess, d3 might be one of the more popular options. At work I'm working on a graph drawing library and we have quite a bit more layout code; heck, for a hierarchical layout (akin to dot) you'd need about 4 or 5 MiB (no idea how well it gzips right now, though) of JS. Good layout algorithms can be complex, and thus large.

Hmm. Downloading the current viz.js from the releases page (v1.4.1) gives me a file that compresses down to 632K when I use 7-zip on its highest settings (7z -mx=9 -mfb=258 -mpass=15 -tgzip a viz.js.gz viz.js).

I've been using http://www.webgraphviz.com/

The hairball problem sometimes has solutions. this is a filtering soltuion which works with all general layout methods: http://visone.info/wiki/index.php/Backbone_Layout

It also scales for large graphs

Graphviz is great. I did a project in undergrad that used https://github.com/jrfonseca/xdot.py to render data structures into a nice, pretty, visible form.

I use graphviz, https://github.com/anaynayak/aws-security-viz, and xdot.py somewhat regularly to mechanically generate pretty graphs of things like connectivity between EC2 security groups.

It's wonderful and I love being able to use my $EDITOR / awk / sed / etc to manipulate the results.

There is also "vimdot" in the graphviz distribution.

Its really just a hack where graphviz uses inotify() to watch for changes in a file being edited by vim.

I actually had no idea that graphviz was actually a graph description language until I clicked on that link. I had used it before, but as a ruby gem, so I thought it was just a library. So thanks!

DOT is the language, and there are other tools which can work with it as well as Graphviz (e.g. Omnigraffle) - https://en.wikipedia.org/wiki/DOT_(graph_description_languag...

Does a dot->gis conversion exist? Would anyone aside from me want such a thing?

I'd like to render complex graphs with Mapnik, as I do for OSM (which is just a graph after all).

Since you'd need the graph layouted already anyway, I'm wondering what Mapnik would bring. Nice label placement along paths is about the only thing I can think of right now. Or tiled rendering maybe, but there should be easier ways for that, too.

Tiled rendering is what I'm most interested in, but I'm not aware of any easier way to go about it. The alternative would be to code something from scratch... But my graphs look so much like the road network I'm inclined to leverage what's out there already.

I'd be very grateful for suggestions.

I could never use this to make a callgraph in a project. Probably because I have two many lines. How does a .dot file looks for you?

Doxygen ( http://www.stack.nl/~dimitri/doxygen ) automatically creates callgraphs from sources code using graphviz.

A good intro is at: https://romanegloo.wordpress.com/2012/03/29/generating-a-cal...

Unfortunately not supporting my language.

So this is a JS client that calls a C file that calls graphviz libraries? And it renders the output SVG? Very cool.

I think it's all compiled into JavaScript with Emscripten.

Support for this would be very useful in a Jupyter notebook.

Wouldn't Jupyter do most of the heavy lifting on the server side so it would be more fitting to render with graphviz there?

Definitely. Jupyter makes it very easy to display SVG, which graphviz (naturally) targets out of the box - as an example, I've used this before in a project to display Python ASTs ( https://github.com/hchasestevens/show_ast ).

There is also Graphiz compiled to JS which works: https://github.com/mdaines/viz.js

This is the same project.

Those layouts besides dot are truly horrible.

They all emphasize different things and thus are usually useful for different graphs. dot has a hierarchical layout, circo arranges nodes on circles, neato uses force-directed layout, etc. Of course they're not going to be all useful for the same graph. When preparing our own layout styles demo [1] we took care in crafting a graph for each layout and also explaining a bit on which layouts are good for which use case. There isn't really a magic layout algorithm that works on every graph to make it clearer. We have something built into our graph editor that analyses the graph and picks a layout that might be suitable, but that still errs often enough since it can't know about labels or what the graph actually represents.

[1] http://live.yworks.com/yfiles-for-html/2.0/layout/layoutstyl...

Reply on news.ycombinator.com