D3 3.3 Released | Hacker News Reader

D3 3.3 Released

github.com

166 pointsmbostock57 comments

I find D3 to be frustrating because it promises so much and yet it's less of a library and more of a "way to write JavaScript". There really isn't an abstraction so much as a convention for calculating the 3 subsets of an intersection and associating each subset with DOM elements, and then 'doing something' with those DOM elements, possibly as a function of the associated data item. Mike has created some amazing visualizations with D3, but for each viz the code is fairly long and hairy. The worst thing though is that it's not composable - or rather, the unit of composability is very low level. D3 doesn't have any helpers for data that changes over time, for example.

As a programmer I'm a hacker--in the golf sense of "someone who spends a lot of time at something but still sucks." But d3 has allowed me to build several useful data visualizations. The code may not be short and sweet, but it is pretty easy to write and pretty easy to understand. Perfect for someone looking for just the right amount of abstraction.

Here's something I wrote a while ago using d3 for my own use. It's entirely client-side js. http://neuvc.com/labs/vcbar/ It charts the investment activity of VCs by pulling Crunhbase data. I would not have written this if I didn't have d3 to use as a crutch for the bar charts.

> but it is pretty easy to write and pretty easy to understand

I wish I was as smart as you because D3 makes almost zero sense to me. I think there's something I'm missing that, once it clicks, will change things. <fingers crossed/>

It's not a question of smartness. Take a working example of code and make it do something else. I'm sure I took some bar chart code from one of Bostock's examples and used that as a base. After a few hours of struggling with it (and with freakin' Javascript) and trial and error debugging, it became clear to me what d3 wanted me to do. Then it was easy.

In my experience D3 is very much like that. Before you wrap your head around the way its data binding model works it seems totally arcane (esp. if you are like me coming from using other visualization tools, like matplotlib, where the hello world example is just "plot(x,y)"). Keep banging your head against the wall, it will eventually crumble and all will become clear.

The fact that it's like a functional set of transformations, using closures?

Perhaps you're thinking to imperatively. Read a simple D3 tutorial and things will click.

I run into A LOT of problems trying to make my own charts, but it also serves to show in many cases that you can't abstract charting to account for all the use cases, because there are so many distinct characteristics in every graphic and dataset.

If you want a higher level of abstraction for d3, try Chartbuilder, which I find doesn't cover anywhere near my use cases: https://github.com/Quartz/Chartbuilder.

Another tool is Bret Victor's proof-of-content editor: http://worrydream.com/#!/DrawingDynamicVisualizationsTalk.

By the by: Magic Maps is unbelievably awesome to get your map nerd on. (Or playing around with any shapefile, really.)

Check it out at http://magicmaps.evanmiller.org/gallery.html.

Did he release a tool or source for what he uses in the video?

    I expect to write a full research report at some point 
    (at which I'll make the research prototype available as
    well).

http://worrydream.com/DrawingDynamicVisualizationsTalkAddend...

Bret Victor is famous for tantalizing people with the future but not releasing code :)

Yeah, I'm not holding my breath either. I think his prototypes are usually just conceptual and not meant for use cases, which makes his talks a bit too unfair and utopian for my taste.

I think the secret to d3 is that before you can get much out of it, you basically have to be comfortable with hand-writing static visualizations in SVG. Once you can do that, d3 can give them life.

I've found D3 to be tremendously powerful for some types of visualization. For more-traditional types of plots, I usually use Flot instead. http://www.flotcharts.org/

http://www.highcharts.com/ is worth mentioning also, really awesome high quality charting library.

>Mike has created some amazing visualizations with D3, but for each viz the code is fairly long and hairy.

Huh? I find D3 code to create a visualisation be much much smaller when comparing with competing implementation of the same visualization.

But then, my metric of comparison is creating visualisations from scratch with imperative drawing APIs (on Canvas/SVG).

Perhaps you want a one-line function invocation to draw the whole thing?

D3 is not for that, it's a plotting HELPER library, not an ready-made collection of plots.

That said, it's not difficult to adapt individual plots made with it.

I'm comparing D3 with two of it's progenitors, Prefuse and Flare. All of this is coming out of Prof. Jeff Heer's viz group at Berkeley.

Thank you for posting this. I had a lot of issues with D3 where it seemed like it wanted to be a domain specific language, but at some point decided to be a reimplementation of JS around some arbitrary starting point of thinking in R-like or HDF-like data frames or something, and then just got out of hand.

There are a ton of cool interpolation and features and such, and you can use them probably here and there without everything else from D3, and I have managed to do that on occasion. If someone were to make a debugger specifically for D3 or just generally say "forget JavaScript and drink the coolaid and just copy our awesomeness and forget about making your own" I think that would be in line with how D3 is presented. The composable part is spot on. I think OpenFrameworks also suffers from this. They demo their awesome thing and eventually make a library in OF, but as far as using OF itself to make that original awesomeness, that's when one would just say "well, I guess OF is cool for doing exactly what you see people doing with OF but anything more is just C++" ... anyway. So it isn't a framework or an abstraction per se, but a series of just previously developed functions that let you repeat what the author did.

Personally I feel the reach is too broad to be considered a library. It's becoming some of an Ext for visualizations.

D3 is definitely a library, though not a framework. There are conventions, but there are also magical abstractions that work together like selections, data-binding and transitions. I would not call the eight lines of code you need to write for a basic visualization and transition "low-level".

Composability is achieved with configurable functions. Is there an alternate solution you propose? What helper would be useful for data that changes over time?

svg.js is a library. d3 is definitely a framework.

This is an interesting comment to me because I came to D3 having written zero Javascript and picked it up as I learned D3. D3 does feel like both a library plus a set of conventions. That being said, I find it to be very powerful once you grasp the basics. I've also used some higher level libraries like NVD3.js when I didn't feel like starting from scratch.

D3 is an absolute blast. I'm continually impressed by it's accessibility and flexibility. If you've never played with it before it's an absolute joy to work with.

Make sure to check out Mike Bostock's (d3's primary author) absurdly large collection of examples: http://bl.ocks.org/mbostock

I second this sentiment. Transform based zoom is a great new feature, now I don't have to roll my own.

I've never had the opportunity to do much with D3 myself, but one of the things that strikes me as really neat about D3 is that the data visualization is aesthetically pleasing (use of colors, transparency, stroke weight, etc.). I love to just sit and look at D3 documents because I often find them to be very soothing and enjoyable as visual elements, independent of the data being presented.

Conversely, I very rarely see any demonstrations of non-Web plotting/visualization tools (MATLAB, matplotlib, gnuplot, etc.) that look nearly as nice.

What's the reason for this? Does D3 look that nice out of the box, or are D3 users/demonstrators just more interested in flipping the appropriate bits and writing the code and doing the work to make sure that their data looks nice? I ask because I'm about to start playing with a personal project that isn't web based and that needs plotting. I'm seriously contemplating embedding a WebKit panel and using D3 to render the graphics, but I haven't had the chance to really invest a lot of time in studying plotting options. D3 has definitely given the best first impression though.

Mike Bostock has a great design sensibility, but there is also some work behind the scenes. It's a little like the LaTeX of charts that way.

Notice how the ticks look, which is dynamically handled, for instance. There's some tweakage behind how many ticks to display, where to place ticks, and other Tuftian goodness.

D3 is not a graphics library, so it's certainly not D3 itself making anything look nice. D3 enables you to easily build great visualization, but it doesn't actually draw anything directly or provide you with a theme or anything like that.

D3 pure and simple binds data to DOM elements and keeps the relationship updated, but nothing more. Those DOM elements could be pure text, or tables, or whatever else you want. It just so happens that D3 works well when bound to SVG elements, but that's purely up to you.

A few reasons:

D3 has built-in components to draw small details like axes precisely, which lower the barrier to making good data graphics.

Also, the feedback loop in my experience is much tighter than with tools like Processing, R or Matlab, since you can manipulate elements directly in your browser, instead of having to compile code/generate a new raster image every time.

As someone already said, Mike Bostock has a great design sense. But more importantly, he has built up a massive set of ready-to-use visualizations. Most others, like me, use his examples as a base to build our visualizations. We often reuse the same properties used in the examples. For example, most circles have a contour of steelblue with a stoke width of 1.5 pixels, text nodes are all sans-serif of 10 pixels, all paths (lines between nodes) are colored #ccc with a stroke width of 1.5 pixels. These are pretty good defaults.

What is D3?

D3.js is a JavaScript library for manipulating documents based on data. D3 helps you bring data to life using HTML, SVG and CSS. D3’s emphasis on web standards gives you the full capabilities of modern browsers without tying yourself to a proprietary framework, combining powerful visualization components and a data-driven approach to DOM manipulation.

I was completely lost without your comment. Thank you

I'm still completely lost. How is a "document based on data" different from any other document?

The “document” here refers to the W3C Document Object Model (DOM): the hierarchy of HTML or SVG elements that comprises a webpage. And “data” refers to arbitrary JavaScript data structures, like an array of numbers or a parsed CSV file. So, D3 is a JavaScript library for transforming the DOM based on data.

http://mbostock.github.io/d3/tutorial/circle.html

The above is an excellent example of both what d3 is AND how it works.

Seems to be broken on Chrome (Win7 x64). Clicking on any of the Run buttons does nothing.

"Data" as in "datasets", not "paragraph data" or "JPEG data".

Highly recommend D3. I've used it for a large-scale project this summer, handling thousands (millions, even) of data points in the browser. Mike Bostock has created an immense body of examples and is constantly found answering questions on SO, etc. Kudos to Mike and D3 as a whole.

EDIT: oops, typo

His name is actually Mike Bostock, not Mark :)

Not a D3 expert myself so hard to know how "big" each of the bullet points are...but the interpolated zoom is pretty slick: http://bl.ocks.org/mbostock/6301817

And custom geometry transforms: https://github.com/mbostock/d3/wiki/Geo-Streams#wiki-transfo...

The slickest part of that demo is the continuous level-of-detail rendering. That's extremely helpful when scaling up to large maps.

Brush transitions are a nice feature http://bl.ocks.org/mbostock/6216724

D3 is the reason I got into animation in the first place. The general ease-of-use combined with the hackability of it is awesome.

D3 is great - It has made a lot of things super easy to make pretty (mostly pretty realtime front ends to research demos). I wish they would add some more support for canvas drawing, but other than that, everything about it is just slick.

What is your specific use case for using Canvas along with D3?

Right now everything I want to do is capable of being done reasonably well in SVG, however, there has been talk of effects that we wish to add that are easier done in canvas.

We are doing a graph diagram using force directed layout, and are displaying various overlays of paths through the graph based on input data parameters. As I mentioned, this is currently all fine in SVG, however, we'd like to in the future be able to make those overlays a bit more dynamic in the graphic sense - by adding some effects that canvas makes easy. Also, our future drawing of the graph will include enough custom shapes and images/sprites that the canvas context + drawing objects will result in cleaner code than the equivalent SVG operations.

Also, and this part is a personal bias, I just dislike SVG and it's DOM for runtime graphics manipulation, and prefer the canvas API for it.

Thanks for the detailed comment. As you might imagine, I think the killer feature of D3 is the binding of data to the DOM and all the awesome manipulation that is possible (via CSS/javascript/jquery etc.). It also happens to scale very well for us. So I look for examples where Canvas is a better fit.

We are doing a graph diagram using force directed layout...

Can you provide a link to this?

Sorry, can't - it's internal only right now. But the demo from the D3 page was our starting point.

http://bl.ocks.org/rkirsling/5001347 is a cute example I came across recently, if I'm understanding the description correctly.

Seconded--canvas support would be great. I too had to mix D3 with some canvas drawing. The specific use case was that I had a number of diagrams, most of which were handled by D3, but one of which required pixel-by-pixel rendering (i.e., each pixel in the canvas was colored according to some data).

canvas can better handle drawing thing which involve tons of objects, as you're working with pixels instead of DOM elements. here's an example of canvas being used for parallel coordinates: http://bl.ocks.org/syntagmatic/2420080

A bit off-topic, but it bothers me (positively). How in GitHub you can provide that kind of tag summary? Is it part of tag message? (There are even links, so I doubt it.)

This is a release, a relatively new github feature

https://github.com/blog/1547-release-your-software

Check out the list of D3 releases https://github.com/mbostock/d3/releases/

Thanks, somehow I overlooked that.

> You can also attach binary assets (such as compiled executables, minified scripts, documentation) to a release.

Nice. I knew they didn't turn Downloads off for good (API still works, or at least worked some time ago), but I didn't expect such move.

GitHub Releases: https://github.com/blog/1547-release-your-software

Are you asking about github releases? https://github.com/blog/1547-release-your-software

What's new in 3.3?

- Brush transitions http://bl.ocks.org/mbostock/6216724

- Tools to help dynamic simplification (scale appropriate geometry on zoom): http://bl.ocks.org/mbostock/6252418

- Bug fixes

- Others?

Reply on news.ycombinator.com