D4 – Declarative Data-Driven Documents | Hacker News Reader

D4 – Declarative Data-Driven Documents

d4.js.org

263 pointsjoelburget79 comments

This is a good start but it glosses over the hardest part of making React work with D3, which is animations. Most animations you want to do with D3 don't fall neatly into categories you can use with CSS animation (namely updating paths and entering / updating / exiting animations).

React as an ecosystem is pretty fantastic but the React animation story is still pretty terrible unfortunately. "Portals" / "reparenting" is a majorly hacky way of getting an element from one part of the DOM to another, and even React Motion, which solves a few of the animation gripes, is hard to use, slow, and brittle in my experience. There just isn't a good substitute for the enter-update-exit selections that make up the core of D3.

My usual workflow is to get as much DOM building done in React as I can, with D3 filling out the tricky bits. D3's lifecycles and direct DOM manipulation are much more complicated to reason about than React, on the whole.

I've tried both, but between D3's odd imho data-binding and React animation limitations, I gave up and found using RactiveJS with SVG directly was much easier: http://examples.ractivejs.org

RactiveJS uses virtual dom as well, so was quite performant and very easy animations of properties.

I don't agree that D3's data binding is especially odd. It isn't as straightforward as React's vdom descriptor, but it's not especially complicated: https://bost.ocks.org/mike/circles/

You are in the minority. Having taught it, I've seen students struggle.

Creating markup through chained functions executed as many times as there are items in an array, where the convention is to name the array singularly:

ie., node.append('g')

is counterintuitive.

It's definitely a steeper learning curve than simply creating a node and appending to the DOM (the d3-specific idioms are particularly obtuse) but it's not insurmountable.

To each their own - my issue is if you are starting with some JS data (via API, etc) - trying to bind to D3 is rather complex compared to using other view tech.

That's fair. Keep in mind that d3's major use case has historically been for building interactive data visualizations with dynamic data. Viewed through that lens, its data binding is a pretty integral part of the design.

Has anyone worked on these issues in Om or Om Next? I feel I'm coming up against exactly this wall myself, and I don't think my worldview is sufficiently Ommy to know which way to go.

Have you seen Samsara?

http://samsarajs.org/

I'm hopeful it could do some of the middle bits.

I haven't but it looks cool. Can you share how it'd solve those problems?

You can use it to move things around the screen more efficiently. It's good at "drawing rectangles" at 60fps.

If using Ember instead of React, Liquid Fire is pretty incredible and easy to use: https://github.com/ember-animation/liquid-fire

I'm afraid I fail to see how Liquid Fire competes with D3 in any way. It looks like a library for UI state transitions, but not for animating graphs, or all the crazy things one can do with D3 (some of these examples are even given in the D4 article).

Sorry, I don't get it. The code in any of the given example comparisons has no less obvious complexity and just removes the abstraction that writes the SVG nodes for you with having to explicitly write them in your JS which I similarly have never understood why we're intent on going back to.

Different for the sake of different is fine, but "preferable" is a bold statement that appears to have no evidence.

I suppose that everything is a matter of opinion, but d3's syntax is the worst combination of obtuse (".enter"???), cute (the incredibly obnoxious chained api), and cumbersome (all the mutation, and the fact that d3 code always ends up being a long string of procedural spaghetti).

Of course, it does have a lot of useful code around math, projections, etc.

This project looks like a very welcome step in the right direction, showing that d3's great math can be used with obvious, declarative syntax.

Can you explain why do you feel 'enter' is obtuse? I found enter/update/exit pretty logical. I do agree that long-term value of Bostock's work is in other parts than data-binding, but I didn't find enter/update/exit particularly hard

I agree with you as well. The enter/update/exit cycle is actually very intuitive although the documentation could do a better job of explaining what the functions are doing to the DOM.

In this case, I think a little more verbosity in the names of these methods would assist in understanding significantly. For example, enterAnimation/enterTransition would increase the clarity for someone new to d3, or who has been away from it for a while.

enter is used to insert elements into the DOM; if there is data in the data-set which isn't represented in the DOM, then new elements will be inserted which have that data. It doesn't have to do with animations really, though you can animate those newly inserted elements if you wish.

I understand how enter works, but my point is that the naming is pretty poor. If I haven't been around d3 for a while, it becomes very easy to forget how it works because of exactly this. Its api isn't intuitive, which often indicates either bad naming or design - in this case, I would pin it largely on the naming, and the api design not facilitating understanding as easily.

are you objecting D3's API lexicon, or the entire notion of how it abstracts data, and binding?

".enter" actually makes sense in terms of describing how data flows through the D3 processing pipeline.

For me, the biggest problem is that d3 is a declarative library which you write code procedurally for. If you look at d3 code, it has lots of imperative-style verbs (enter) which are not readily clear.

Quite simply, there's an impedance mismatch.

Sounds like an opportunity for applying Ohm/OMeta?

+1 I couldn't express it better.

I agree with you that is his personal opinion about "preferable" way to use D3.

> "Unfortunately, I always find D3 code surprisingly difficult to understand and extend"

I would say something different - it is very easy to extend and create various visualization. It is easy to scaffold new chart (line, pie, box-plot, heatmap etc.) and customize it to specified needs. I'd rather say that it is difficult to create a brand new type of visualization which wasn't been already developed [1] and it is easy to understand by random folk.

[1] https://github.com/d3/d3/wiki/Gallery

Yeah. I think the premise is reasonable - d3 can be a bit of a pain, but I'm not convinced this is the answer.

It just makes it more declarative. That's all. One can continue with their preference.

Are you saying the react way is more declarative?

Of course it is! to start with, it looks like SVG, but also you only have to provide the state that you want and React adds and removes what is necessary, while in D3 you need to use `enter` and `exit` imperatively.

I'm just concerned about speed. Diffing works ok for forms but charts with 1000s of elements... let's see.

As someone who has worked with D3 and React a lot, D4 makes a lot of sense.

And the name is great!

enter() and exit() are declarative. You're not causing anything at all to happen, but rather declaring what should happen, in the form of a function, when new data is found or data becomes stale.

Declaring 'what' structure should exist

- vs -

Declaring 'how' this structure is created

D3 is more declarative than jQuery, but not so much as React.

selection.enter().append("div").attr("class", "bar") .style("height", function(d){ return d + "px"; }) .style(“margin-top”, function(d){ return (100 — d) + "px"; });

enter is not imperative, but what you write just afterwards is.

It sounds like the difference is just syntactical. Is this true? I have no experience with react, so I wouldn't know.

The problem is that calling a function called "enter" does not feel like writing declarative code.

It's an active verb and seems to imply an imperative style. Except it's actually declarative. The problem with d3 is that it takes a declarative system and gives it an imperative-looking API.

I think you're reading it wrong. `enter` isn't imperative. You're asking for the `enter` selection, which is the list of entering nodes. Same thing with `exit`. Calling either of those methods doesn't actually alter any data

I know enter isn't actually imperative. That's the problem: it looks like it should be.

If I write "node.enter()" it definitely looks like I'm making the node is being made to enter if you aren't well-versed in d3.

And SQL `SELECT` statements looks like I'm actively selecting something, when in reality I'm declaring what bits of the table I want returned... All syntax looks opaque until you understand the underlying concepts.

Uh, a SQL SELECT statement is actively selecting something. You're selecting what you want to return. Terrible example.

The problem with "enter" is that nothing will happen if you simply use enter by itself. Yet it's an active verb which seems to imply that something will be made to enter.

SQL SELECT declares what will be selected when the query is run. The query engine does the selecting, including figuring out how best to do it.

Yes, SQL is declarative.

But you still "SELECT" something to look at. A SELECT statement is, by itself, sufficient to get something to happen (for results to be fetched).

On the other hand, a D3 enter() does absolutely nothing by itself. You must do further calls for it to have any impact.

This is very off topic and ridiculous.

OK, if a SELECT statement is sufficient by itself, surely you can tell me the results of this:

    SELECT * FROM stuff WHERE thing=2

Of course you can't. Because every declarative process has two main steps: declare and execute. SELECT and enter() both only declare; they must then be executed against data in an engine to produce a result.

What we mean when we say "SQL is declarative" is that it doesn't provide instructions on how to carry out a task, and instead only provides a description of the outcome of the task, once it's been carried out. SELECT is declarative because it doesn't tell the computer how to do a query. SELECT tells the interpreter what you want not how to get it.

It might be proven to be more performant, as it's relying on React's virtual DOM rather than d3's presumably made-from-scratch data binding. Would love to see some benchmarks though.

Hybrid approach:

> let React do all the migraine-inducing heavy lifting to figure out what to enter and exit, and have D3 take care of the updates.

https://medium.com/@sxywu/on-d3-react-and-a-little-bit-of-fl...

Love this - awesome work.

Wrote an article on this concept last year: http://formidable.com/blog/2015/05/21/react-d3-layouts/

and spoke about it at Reactive2015: https://www.youtube.com/watch?v=n8TwLWsR40Y

Have been doing data visualizations like this ever since, it works. Yes, the hardest part is animation, which we had to address, as well as the axis components and other higher order functionality that builds on d3 itself:

https://github.com/FormidableLabs/victory-animation

http://formidable.com/open-source/victory/docs/victory-axis/

My team and I have been using and finding success with such a pattern for a couple of years now.

As others are alluding to, transitions are not currently so easy to express with React alone. We wrote and recently open-sourced a React component that aims to encapsulate the power and simplicity of d3 transitions (feedback & contributions welcome): https://github.com/dechov/join-transition/

Nice - thanks for sharing this

>>There are some pieces of d3 that I would love to use but aren't easily portable. For example, d3-drag and d3-zoom smooth over a lot of the quirks you'd have to deal with when implementing dragging and zooming, but they're only designed to work with d3 selections [...] Ideally we could decouple these libraries from selections, but it might also be possible to mock the selection interface they expect while still using React.

According to the changelog for the recently released version 4.0.0 of D3.js by Mike Bostock[1], one of the big shifts is for D3 v4.0.0 to be "composed of many small libraries that"[2] can be used independently. So the author of D4.js can now more easily examine Bostock's refactored code for Zoom and Drag as they please.

[1] https://github.com/d3/d3/blob/master/CHANGES.md

[2] https://github.com/d3/d3/releases/tag/v4.0.0

If you look at the example code, the author is aware of the new modules pulled out of d3. He's actively using them. It doesn't change his comments re: those two modules specifically.

I find that D3 is great, but it needs an abstraction for the general update pattern (https://bl.ocks.org/mbostock/3808218).

I did that! One function that receives actions for update, create, delete, etc. Now my code is easier to read. Check the function gupChildren (AKA: General update pattern for child nodes of a selection) in my code, and feel free to use it: https://bitbucket.org/aurelito/sandro-lib/src/83b81c4b556848...

At last, unless you understand what you are doing, use selectAll and no select.

Cheers!

argh.. I hate the style of mixing JS with markup language....

Well its actually function calls that look like markup language. Which is why you can write the function calls yourself if you hate it.

Me too, so last week I created gulp-pug-hyperscript to compile Pug/Jade templates to Hyperscript [1].

In order to completely avoid JS in the markup, event binding must be done outside of the markup. I do this by using event delegation rather than binding directly to elements.

[1]: https://github.com/nextorigin/gulp-pug-hyperscript

Agreed. Please tell me there's another syntax for this. Please don't tell me that we're stuck with this sad relic of the past.

There is. You can write regular function calls like this:

  React.createElement('h1', {}, 'Hello')

Personally I like JSX a lot, but you don't have to use it.

I prefer:

    h1('Hello')

https://github.com/Jador/react-hyperscript-helpers

I'm not sure if this is it, but the world needs a simple way to integrate D3 and React. Looks good at first sight though.

Here's a library that integrates the two https://github.com/react-d3-library/react-d3-library

That's actually a pretty reasonable idea, considering that we have several fairly different renderers for React already: DOM, Native and Three.js (http://izzimach.github.io/demos/react-three-interactive/inde...).

The big question is, should it render down to D3's API, or should it implement it in a way that is idiomatic to React, like this demo?

From the site: This is not a library, but rather a demonstration that it's possible (and preferable) to use React instead of the core of d3.

Shameless plug: I have been proposing this approach for long. I found that the missing piece is how to generate the actual geometric information starting from data, hence I wrote a library to this effect https://github.com/andreaferretti/paths-js

For the animations, it is often just a matter of continuosly updating the state of some component, something that I did in this mixin https://github.com/andreaferretti/paths-js-react-demo/blob/m...

I've done the opposite: use d3 to provide React-like semantics.

- Basically the data fetches get pushed to a lightweight queue/topic system in the browser (using publish.js).

- Each part of the page listens for messages it cares about and updates the page accordingly.

There is still some of the d3 plumbing (I must specify how updates differ from mere element creations, and handle the exits). But I still get a nice decoupling between the different parts of my app. Coupled with browserify, this ends up with semantics not unlike Erlang (I even named the listener method "receive" and it acts on patterns of messages).

I'm excited to play with this. I wonder what kind of overhead it has. I imagine it would be fairly low.

My CPU fan begs to differ. Or maybe you meant something different by overhead?

Intelligently-optimized JS animations are generally more performant than CSS ones by an order of magnitude. See Greensock for relevant examples. Side note, your examples don't assign keys to your array-mapped paths, which will emit warnings.

@joelburget, btw the "demos" link leads to a 404.

( https://github.com/joelburget/d4/demo )

Very cool.. My only question is.. Is this "D4.js" a full feature replacement for D3.js? Could be a bit misleading. Might want to have a new name for it? Thoughts?

The very second sentence of the article answers your question:

This is not a library, but rather a demonstration that it's possible (and preferable) to use React instead of the core of d3.

Yes. So then the title is very misleading and should not be called D4.js.

Did the title get changed? Both on this post and the article, there's nowhere that it says it's D4.js.

I've used the d4 pattern, but I've referred to it as d3act, as in "dee-three-act".

I've been working with a similar approach for a while and I give it my vote of confidence. Much more elegant than d3's awkward data-binding.

d3.transition works on element attributes not just CSS. Wish React had something like that!

You can do that easily with https://github.com/chenglou/react-motion

I've played with ReactMotion, but I was under the impression that it was only for styles, not for arbitrary component attributes.

In the Readme they say the `style` property is required, and they also don't show any examples of animating something other than style.

It gives you a continuously changing variable that you can use for whatever you want. The name style is confusing, because it isn't actually a style. In the very first example it's just displaying the changing value.

What is this? I can't even scroll.

I need something exactly like this.

Check out:

http://formidable.com/blog/2015/05/21/react-d3-layouts/ http://formidable.com/open-source/victory/docs/victory-axis/

Reply on news.ycombinator.com