Hacker News Reader

Comment by warpech | Hacker News Reader

Technically, D3 is a JavaScript library, but in reality it is much more than that. In this article, it is called "visualization grammar". I have heard it being called "jQuery of diagramming", "declarative DSL for data visualization" and so on.

Since 2011, there were already numerous cycles of the best of breed JavaScript SPA frameworks and libraries (I'm talking everything from jQuery, through AngularJs, to React), but in data visualization, D3 seems to hold its position very well.

What are the things that D3 did right to get the acclaim, and why does it keep it for so long?

D3 is approaching 10 years since the initial release, did it stand the test of time and will it keep its power for the next 10 years?

View on HN

> What are the things that D3 did right to get the acclaim, and why does it keep it for so long?

Once upon a time, there was an infovis library for Java called Prefuse. Then the developers abandoned that library for a Flash infovis library called Flare. A JS library called Protovis was inspired by Flare, and then Protovis itself was replaced with D3.

I've used 3 of those libraries (never wrote any Flash code). The only one I would actually recommend people use is D3--it's literally the first (honestly, only) infovis toolkit I've used that doesn't feel like pulling teeth to get anything done.

What D3 does differently from its lineage is that it is data-centric. Each datapoint is an object, corresponding to a DOM node, and the layouts in D3 don't actually draw anything; they just set properties on the object--you're responsible for actually converting those into setting SVG properties for display.

It sounds like this would be a nasty headache, but this way is actually much better than the traditional way of handling things (where the visualization is essentially treated as a display widget in a normal GUI toolkit). Any time you want to progress beyond a basic "display a graph with static data," you start to need a lot more fine-grained control over display elements. Want to link multiple displays of the same dataset, so clicking on one in one screen highlights them in all the others? That's very hard in prefuse, but quite easy in D3.

Conversely you often seeing D3 recommended for people that just want to draw a bar chart. That's a big learning curve and a steep cognitive overhead if you do just want to do something fairly conventional.

Luckily there's now plenty of d3 wrappers to cover common cases.

if you have experience, witch 3 would you recommend for business as usual presentations ?

Vega-lite is nice: https://vega.github.io/vega-lite/

They kept it really low level and didn’t make many opinions on how to join the dots. There are functions to manipulate data, functions to make scales, and functions to and functions to render svg (plus dom bindings). These decisions make individual charts quite verbose, but the trade off is you can visualise anything you want. Just look on npm how many charting libraries use d3 as a base.

This is true. We decided to use D3 directly to render charts at my job, and it was honestly kind of a terrible decision. Ideally you would use a library that uses D3 to render charts.

What made d3 a terrible decision?

My guess is that d3 is too low-level to make reuse simple, and that using a higher-level abstraction would make the code more maintainable.

My guess is that d3 is too low-level to make reuse simple, and that using a higher-level abstraction would make the code more maintainable.

In what context? It's not that hard to collect some scales and axes into a JS object, mix it with judicious use of CSS et voila.

I understand that d3 is low-level and verbose, but I can't say that I've any regret over deploying it in production.

It's too low level. For reference, I just re-implemented the entire thing in React. just react, with svg elements directly. Not using D3 at all. And it was roughly the same level of effort.

edit: saying that D3 was a terrible choice is not saying that D3 itself is terrible. D3 is amazing. But before choosing it, you should know exactly what it is, and whether it helps in your specific use case.

That's the idea when you're using D3 with React: React handles the DOM updates, D3 handles data wrangling and math. You said you reimplemented your charts in SVG directly – did you also reimplement scales, chart axis generation, d3-collections, d3-time, etc?

I'm not sure how you'd implement scales, color theory, etc.. in React (a rendering library). There are a lot of useful parts of d3 which have nothing to do with rendering to the DOM. In fact, React and D3 complement each other quite well (maybe that is what you were saying?).

D3 is slightly more verbose than the one liners, but with that I get complete control over the scales and appearance of my chart. In addition it's trivial to combine multiple charts in a single diagram.

Out of interest, how did the performance compare between the React+SVG and D3 implementations? (and what sort of data size / structure were you working with?).

> D3 is amazing. But before choosing it, you should know exactly what it is, and whether it helps in your specific use case.

I agree. D3 is really great, but not always the most appropriate solution.

each D3 project is an entire application of its own

adding the “simple pie chart” to your view is not simple at all.

It needs independent UX planning, system architecture planning, and multiple sprints to implement

All the example code and prebuilt charts are outdated due to using old versions of D3 or other dependencies and essentially have to be rebuilt from scratch

All the example code and prebuilt charts are outdated due to using old versions of D3 or other dependencies and essentially have to be rebuilt from scratch

As part of moving them over to observable (which I'm not a fan of) the examples are being updated to use the v5 API.

I had the same experience. It started great when you just want a simple line chart, but when you need more and more features, it will be like creating tons of abstractions and writing a whole charting library ...

Same here, too much work just for simple charts

What are the things that D3 did right to get the acclaim, and why does it keep it for so long?

d3 (especially now that they've broken the bits and bobs into discrete packages) takes a very unixy philosophy: do one thing and do it well. d3 lets you iterate over your data and get useful items back (e.g. scales that you can use to create geometric shapes for a chart). It's intimidating for some but offers a ton of flexibility you don't see with other charting libraries.

Mike Bostock also put a lot of effort into making sure that d3 lets you create beautiful charts.

Every other charting library I've seen takes a very different approach. Often enabling you to create ugly, inflexible charts.

In addition to breaking it up into discrete packages, the devs are also OK with improving things even if it includes breaking changes.

In my mind, D3 is a DSL for ingesting structured data and outgesting DOM. While it has certainly been a few years since I touched it last, what I remember was that it basically didn't touch anything in the data or DOM, it just made the piping between the two super easy. So I think it was that approach: not introducing new idioms for existing things, but instead introducing the pipeline approach, with a very simple syntax, all made it very amenable to learning very quickly and rapidly iterating an idea, tweaking it until you get just the thing you want.

> What are the things that D3 did right to get the acclaim, and why does it keep it for so long?

As a dev, what D3 did right for me is kept the code stable, provided clear sample code for most use-cases, the internet (thanks folks) made lots of runnable examples available via blocks and jsfiddle, maintained precise documentation (even for older versions in github as markdown files) and has almost no external dependencies (so my local .js+.html files from 3 years ago still work without needing any broken npm install or jspm install)

Moving from v3 to v4 was not as painful as I thought it would be.

Once I got over the core concepts of selections and enter/update/exit cycles it has been a breeze and a pleasure to work with.

Thanks Mike and everyone else who's made reasonable and sensible choices for D3 !

Vega calls itself a "visualisation grammar" and I think it is much more worthy of that title. I just wish it was efficient. Sadly it represents every data point as a JavaScript object. In other words it uses array-of-structs style rather than struct-of-arrays which is an obviously bad choice in JavaScript if you want efficiency.

Interesting what you say about struct-of-arrays, I have been following this topic closely recently. Do you know any well known JS code implemented using struct-of-arrays or have some interesting resources about it? I am worried that SoA codebase is difficult to maintain and hard to express in a simple API.

I have been reading two resources about it, https://ourmachinery.com/post/data-structures-part-1-bulk-da... and https://stackoverflow.com/questions/39799874/why-does-javasc....

Here is a notebook on the subject. I’d also check out Apache Arrow.

https://observablehq.com/@mbostock/manipulating-flat-arrays

Thank you for that. For the first time I finally understood what the Apache Arrow is - until today, I didn't realize it's just a way to do SoA in different languages + a lot of buzzwords.

It's in a different domain, but you might be interested in unity (the game engines) data oriented stack. Basically it serializes properties attached to an entity, which you then write systems that map against entities with certain combinations of properties attached. Seems like this could be amenable to JavaScript as well.

No, sadly most Javascript APIs seem to use array-of-structs. It is definitely easier to write the code that way. I suspect most graphing libraries start out like that and then by the time they try to plot a heat map with a million points it is too late to fix.

Traditionally plotting libraries like Matlab and Matplotlib work on arrays of data.

D3 does the Right Thing with respect to dataviz.

That means: it's conceptually overloaded for doing quick-and-dirty conventional charts, where you just want to plug in a few parameters and have it "just work" - but excellent once you need to customize and make it work with your specific requirements.

That it has the concepts, and a clear notion of them, is the critical difference. Most libraries, most of the time, don't add new concepts, they just have a premade black box of features and functions. Sometimes you want a premade black box, but often you want to open up the box shortly afterwards, and that creates the inevitable trend towards either remaking it as your own box, or being one of hundreds of people who gradually grow it into a monstrosity that does everything.

But a library that is concept-focused doesn't have to get that much bigger: it's just another kind of interface, like a programming language or an operating system, and that puts it on a more sustainable track.

I think of it as the assembly of data viz.

Personally I think it’s force simulation and geo/projection libraries are the thing that have made it stand out. It seems to be the de facto choice for any mapping or simulation visualisations. It’s DOM selection mechanics are neither here nor there in comparison.

Having done a lot of geo-spatial web projects, I would, hand down, use OpenLayers as my go to over D3. Not that there is anything wrong with D3 and it is a great library but man it's a pain to maintain and extend a D3 mapping solution.

I just ported a D3 solution to OpenLayers for a client and they where over the moon with the speed at which they could implement new features. Mobile touch was the killer for them and made them finally bite the bullet and port, getting a D3 mapping solution to behave correctly on both the desktop and mobile requires a lot of wiring.

"D3 seems to hold its position very well"

Just look how long Processing held its position. People are still using it with P5 on the web, but I think it makes a bad figure compared to D3.

Declarative data binding?

Reply on news.ycombinator.com