Learn D3 | Hacker News Reader

Learn D3

observablehq.com

621 pointsmbostock71 comments

Most intros to D3 (including D3’s own home page, for now) focus first on selections for DOM manipulation. Selections are great, but given how many viable approaches there are today for DOM (React, Svelte, hypertext literals, etc.), I wanted to emphasize the modularity of D3. So selections and joins aren’t introduced until part 7 where they’re framed as a tool for animation and interaction: efficient, incremental DOM transforms. I think that’s a better way of thinking about joins, and hopefully helps people focus first on building effective static visualizations before getting into dynamic graphics.

Thanks Mike for giving us d3!

I was always curious about your thoughts about how d3's paradigm is similar to React/Svelte/SwiftUI/<any other UI lib>.

I feel that d3's approach is similar in the sense that you have some data, and you describe in code how you want the data to be map to a UI (DOM in this case).

With selection/joins it seems that d3 adds an extra capability that allows one to "do something" before and after data changes.

In a way of analogy this would be similar to have finer grained control to when props change in React: d3 is similar in being able to transition specific prop numerical values (or something of this kind?).

Anyway would love a TLDR on what you think about d3 and its similarity to other libs.

First, D3 is not primarily a DOM library or UI framework; it’s a loose collection of tools for data visualization, and is designed to work with other UI libraries.

Second, the main difference with d3-selection (the part of D3 that does focus on the DOM) is that it focuses on transformation rather than representation. You describe changes that you want to apply to the DOM, rather than the desired current state of the DOM. This makes it harder to use (more complexity) than more declarative frameworks (UI as pure functions of state) such as React. But the advantage is that you can control exactly what happens as you change the DOM, which can make it faster, and makes it easier to animate transitions.

Can you comment on why you think d3 is harder to use than a paradigm like react? Isn’t UI also a function of data in d3? When you make changes to the data the tree (DOM) is also updated. That’s why the differences are blurry to me.

I would be curious to see how d3 scales in complexity for reactive UI apps... such as todo lists, forms, pages with buttons, etc.

I’m saying selections are harder than React, not that D3 is harder than React (because you can use React with D3 without selections). And selections are harder because, although they are functions of data, they are used to apply transformations (deltas) rather than declare representation (state). And as I say in the tutorial, thinking in terms of transformations is most useful when you want animated transitions and to optimize incremental updates. So for example I try to let Observable’s dataflow handle as much state as possible, rather than doing everything in event listeners.

Thank you so much for Observable and D3! Could you tell how it’s possible to render D3 on a backend if it’s possible at all.?

Much of D3 doesn’t require the DOM, so if you want to do things like generate SVG on the server, you can use d3-shape, d3-scale, d3-path, and the like as-is.

If you do want to use the DOM on the server, you could use JSDOM. d3-selection and d3-transition use JSDOM heavily for unit testing and it’s worked well, but I’ve never used it for server-side rendering.

I’ve tried to use d3 without importing d3-selection (in Vue) but I still need it for d3-axis. Would it be possible to make d3-axis more DOM-agnostic?

Everything is possible but it’s work. I’m not planning on rewriting d3-axis to be DOM-agnostic (especially because d3-axis also supports transitions). As the tutorial shows, you can use d3-axis with Observable’s hypertext literal without selections because you’re allowed to embed arbitrary DOM elements in expressions. There are ways to inject arbitrary DOM content when using Vue or React, too, but perhaps not as convenient as hypertext literal.

A custom directive in a Vue template[0], is quite similar to your example with an inline expression in an hypertext literal. Maybe a little less elegant. But d3.select is still needed to select the bound dom element ;-)

[0] https://stackoverflow.com/questions/48726636/draw-d3-axis-wi...

Hi Mike! I really like what you've done with D3. I've been hesitant to include D3 in my personal projects though just because of how complex both DOM manipulation and async data binding through APIs can get.

Most tutorials focus on D3 itself, but I haven't been able to find much on how to properly integrate D3 into a complete web application (e.g. a REST API build on Express/Koa serving data to a React/Vue frontend). Is this an area where you think there is a lack of tooling, or perhaps simply a lack of information? Given how many companies use D3 I feel like I must be missing something!

one thing that might help is this blog post I'm (slowly) putting together:

https://wattenberger.com/blog/react-and-d3

I've done a lot of work that uses d3 within a JS framework (React, Vue, Svelte, & Angular), and the key is to understand 1. how to create shapes in plain SVG, and 2. how to use the parts of d3 that don't do DOM manipulation. I've found the least-hacky way is to handle DOM manipulation with the framework, and use d3 for things like data manipulation, creating SVG path strings, creating scales to map from the data domain to the visual properties, etc.

This is a great resource, thanks for putting this together.

I would first focus on the overall design of the web application (so, look for React tutorials etc.). Treat D3 as a modular, low-level component within that application. You could start by creating static visualizations, say just using d3-array, d3-scale and d3-shape, and only opt-in to d3-selection and d3-transition when you add animation or interaction, if at all.

This is the best and most recent series of tutorials i’ve seen on integrating D3 into React, and it uses up to date Hooks and React Functional Components:

https://www.youtube.com/playlist?list=PLDZ4p-ENjbiPo4WH7KdHj...

Do people sit in front of the computer watch/listen-to these instead of reading something like https://www.newline.co/fullstack-react/ ? I mean - without committing suicide?

If so, ... how?

You probably didn't see udemy or the likes yet, huh? Of course people watch courses. Especially in the world of tech, a good video course will show you what a book cannot (since they cannot contain 10,000s pictures or animation or many details or ...) - with front end this rule applies tenfolds.

I agree, videos are great to convey concepts but text is superiour for step by step tutorials.

I would much rather watch the YouTube video GP posted than look at the ad you posted.

As a beginner, i find that watching someone code and talk provides a lot of context you don’t get from a step by step Hello World guide or API docs. I only turn to docs once I have a reasonable understanding of something. I also don’t want to commit to a 5 hour course, when a few 20 minute videos could give me enough of an understanding that i would then be able to navigate the docs.

In my book (and free github repo) I have a lesson about 3rd-party integration.

https://github.com/kay-is/react-from-zero/blob/master/16-adv...

I used this in a few projects after playing around with wrapper libraries. This gave me the most flexibility.

The interaction with the DOM is what turned me off to D3. The charts, graphs, user interactions, and even the API (aside from the DOM integration) are great. D3 tries to force an opinionated declarative abstraction to the DOM on the developer. I don't have any problems interacting with the DOM and don't need a tool like D3 dictating to me how that interaction should work or how to write code/interfaces outside of the graphs/charts it generates.

But it’s all low level functions so you can use your dom manipulation, right?

Exactly. Here’s a quick example of D3 without joins (and more are in the introduction):

https://observablehq.com/@d3/d3-without-joins

Using d3 (without selections) with react has enabled me to make some awesome visualizations for my react/react-native apps and I just wanted to say thank you. d3 makes it hard to go back to doing similar things in other languages.

I must admit that when I combine D3 with modern general-purpose frameworks I prefer to use the provided views/components as simple containers and use only d3 inside. I love the "transformative" approach (as opposed to the "state description" approach you mentioned elsewhere). It's just too good to "re-join" existing DOM elements with new layout rules to build crazy advanced visualizations. The widespread data/DOM binding pattern is easier for simple stuff but becomes convoluted for more adventurous uses.

Another comment, at the risk of sounding like a crazy fanboy. After years of use, I still can't get enough of D3, it doesn't feel like JavaScript yet it fully embraces the best parts of the language.

It is indeed a lot more difficult to learn than other solutions, but with the experience it becomes second nature to "think in it" and explore data iteratively, building visualizations along the way all while having fun (and its concepts map very nicely with how Edward Tufte lays out things in his books).

All this to say: thank you.

Thanks for the library! My very first professional project was making a choropleth map for the USGS and I used d3 to do it! Hella complicated though given it was also my first javascript project too haha

https://tlbsoftware.github.io/streamgage_costs.html

It’s not very mobile friendly given I had yet to learn about bootstrap!

Ah wow that's a great insight, looking forward to diving into this

It's funny; when D3 was brand new it was selections and joins that were the big new unique feature, the thing that allowed the magic for D3. It's interesting to hear they're still important but that you'd de-emphasize them a bit in favor of other D3 features.

I've used React and D3 a lot lately and while I do use React for DOM rendering - I still think D3's built in DOM manipulation/rendering is better at this specific task. This is especially true when dealing with animations and complex items like axes.

This looks like a great new introduction to D3. It uses Observable as an interactive playground for D3 example code, which I think is an ideal environment for teaching concepts like these. I hope this will help more folks get started in visualization.

If anyone hasn't given Observable a try yet, I highly encourage you to do so. It's more than just "IPython/Jupyter but for JavaScript" — it's reactive, like a spreadsheet. You define "cells" of JS code that are automatically re-evaluated whenever one of their dependencies (another cell) changes.

It's an amazingly productive programming environment. These days, most of my JavaScript projects start off as Observable notebooks because it's such a great platform for experimentation. And I find myself wishing for Observable's features whenever I'm developing in other languages.

I know reactive programming is nothing new, but Observable has been my first opportunity to use it regularly, by "backporting" reactivity into one of the languages I work in most often. Being able to use it for prototyping and visualization for the past couple years has given me a really warm, fuzzy, "what-a-time-to-be-alive" feeling.

A different introduction to D3 was published a few weeks ago by Arvind Satyanarayan of MIT, if you want another perspective. Here’s the link and discussion on that.

https://observablehq.com/@mitvis/introduction-to-d3

https://news.ycombinator.com/item?id=22476930

Thanks for posting. do you have any opinion on NVD3 or other frameworks on top of d3? I always liked the fine grain control plain d3, but thought the idea of out of the box interactive charts to be pretty cool

If you use a chart typology (a tool that implements a fixed set of chart types), you tend to hit a wall with configuration. So I prefer to either fork an existing D3 example, where I can edit the code to my liking, or use a principled visualization grammar such as Vega-Lite.

In my opinion, the Observable introductions and learn-by-example posts[1] are the best way to learn D3. They are very well written and easy to follow. I have a couple books on D3 and: 1) they are all out-of-date because of the changes in V4 to V5, 2) the authors write/use D3 in weird ways. I look at Observable's code examples as the canonical way of using the library. Please keep up the good work!

One suggestion is to have some sort of curriculum or categorization of the posts from easy to hard. For example, I had to skim through 120 posts to find the easy examples (bar chart, line chart, pie chart) then moved on to the more difficult.

[1] https://observablehq.com/explore

EDIT: Oh, i just found this: https://observablehq.com/collection/@d3/learn-d3

Rating the difficulty of the examples is a great idea, thanks! The gallery currently leads with harder ones because they tend to be more interesting (animation and interaction), but you can find easier ones farther down.

https://observablehq.com/@d3/gallery

I've always enjoyed Amelia Wattenberger's approach to teaching D3. Her intro blog post (https://wattenberger.com/blog/d3) actually pushed me to purchase her book (https://www.newline.co/fullstack-d3). I found her on the last D3 post on HN (https://news.ycombinator.com/item?id=22480270).

I'm on the fence about purchasing the book. How do you find it?

I bought it as well and it served its purpose (then I transitioned to Observable). Worth it if you are serious and want some 'hand-holding' getting started.

As someone who has a codebase with a number of custom D3-based visualations: complex D3 could be your future maintenance headache (unless you have someone who really understands D3).

It's so different from other parts of web development that a developer probably can't just jump into the code and fix a bug, you have to understand D3's coding model, and then read the code you've got to build a mental model of how it's translating to what you're seeing on screen (the mental model building is hardly unique to D3, but you're not going to be reusing any of your existing mental modelling skills here). Then you will go back to your normal web-dev, forget everything you learned, and have to go through the whole process again in 6 months when you have to make another change.

D3 is super powerful, and very fun to use, but for production we stick to Highcharts whenever we possibly can. There's lots of stuff you get out of the box from a good charting library that you're going to be building for yourself if you use D3. Though I have been told that D3 v5 is easier to learn and use.

D3 is pound-for-pound probably the most powerful tool for front-end development. However, its imperative interface makes it really hard to use when combined with most modern stacks. Are there any tools out there with similar feature sets to D3 that use a more declarative interface?

The “imperative” part only applies to selections. I encourage you to read the tutorial to get a better sense of the other declarative parts of D3 (d3-scale, d3-shape), and how you can use these parts with whatever UI library you prefer. Selections aren’t even mentioned until near the end of the tutorial.

I tried to learn D3 over the weekend. Observable was one of the worst experiences I have ever had with code examples. Every other line is riddled with console output, which makes it hard to take in the example as a whole. Downloading and modifying the example is even worse. It gives you the compiled output of their GUI coding tool.

I ended up using plotly's libraries that are built on top of D3, and it really made me appreciate how much work they put into making it so easy to use.

I’m so sorry you had a bad experience. We’re trying really hard to make it great! Our entire reason for being is to make it easier for people to tinker with and share examples.

One of the least obvious things of Observable is that it’s not actually JavaScript—it’s a reactive extension of the language. We’re making that more apparent by making the dataflow visible, and we’d love to hear any more feedback or suggestions you’re willing to share.

https://observablehq.com/@observablehq/observables-not-javas...

https://observablehq.com/@observablehq/introducing-visual-da...

No where on the site do I see how to use these tools in my development environment. Even searching the topic produces no results. A quick browse of your GitHub does not lead to any obvious entry point for developing with Observable locally. This feels like a startup has prioritized onboarding people onto their platform rather than providing examples that are representative of how D3js is actually used. I am baffled as to why these are the official code examples.

It seems that Observable is an online environment - it's true that this is not explicitly stated, but it's alluded to throughout the homepage.

EDIT: Check out their about page for an explanation: https://observablehq.com/about

> It seems that Observable is an online environment

The site's interface is a massive turn off. I love using my editor, and if there's a way for me connect it to Observable, I'd use the platform full time.

there's observable runtime https://github.com/observablehq/runtime

I've done a few small personal projects in d3 and I like it a lot. My biggest gripe though, is that I can't find decent online help for it. It's not popular enough to have its own subreddit/stack exchange. So often when I've been stuck, I end up having to find 5 year old stack overflow threads that work about half the time.

Is there a good community for it I don't know about? Great tool either way though.

Our goal is for Observable to support the D3 community, but we’re not quite there yet in terms of collaboration features (e.g., making it easier to comment and ask questions on notebooks). We do have a nice forum, also: https://talk.observablehq.com

The D3 slack is another place you can ask questions if you don’t want to use Stack Overflow. And you’re always welcome to ask me questions on Twitter, and I’ll try to point you to the relevant example or make a new one.

"Slack Overflow" ;)

Having created mostly static in excel, I found the premise of d3 awesome! As the complexity to use it was kind of overwhelming, I ended up using plotly.js and its react wrapper for an analytical app at work.

Whilst this works great and is super straightforward, there's many little things that you can't easily control in plotly (and other similar libraries). They come with some bugs/quirks that perhaps are meaningless to 95% of their users, but limit me from making my ideal dashbaord.

E.g. plotly doesn't seem to support formatting a date as a week with a ISO8601 (%V): https://github.com/plotly/plotly.js/issues/424 and many other little things.

What's the best way to use d3 with react these days, then? Is there some recommendation from Mike or someone else who is experienced in this area?

Repeating a comment I made above, this semi-finished blog post I'm working on might be helpful:

https://wattenberger.com/blog/react-and-d3

I generally like to stick with React for DOM manipulation, and use D3 like a utility library.

> This may feel like cheating, but it’s okay! Examples are not merely reusable templates but tools for learning, hinting at subjects for study. “Breaking” an example by tinkering—changing stuff and seeing what happens—helps you achieve understanding faster than passive reading.

I haven't had to use d3 in anger for some time but have an idea that means I need to re-learn it; I haven't finished yet but this is a great article. Thanks all for writing and/or sharing it.

In my limited experience, I've found that D3 excels interactive graphs / data visualizations. However, when I've tried to use it for interactive math diagrams, I've found that it doesn't save me much effort. I found it was still necessary to manually track state and enforce geometric constraints. (not mine, but here is an example where d3 doesn't help much: https://observablehq.com/@renatoppl/torus-knots)

Does anyone have tips for getting more mileage out of D3? Or using other tools? I'd be especially curious if anyone's successfully used React+SVG for interactive science/math diagrams.

I have!!! I use d3 and react (and react native!!) together with SVG to create the charts on my site here: https://hopewaves.app

I learned from this repo https://github.com/sxywu/react-d3-example/tree/master/src/vi... . I have been using this combo for a while now so let me know if you have any questions about it

D3.js doesn't handle any geometry. You have to tell D3 how to position elements manually, like so: `element.attr('x', value)`

Because of this, D3 might not be suitable for 3d work. Of course, you could do the math yourself to project 3d shapes onto the 2d plane, but you won't find those sorts of helper functions in D3.

Instead, D3 is a collection of tools to help map data to UI elements. It is up to you to use D3's tooling to generate, update, and delete UI elements.

This tutorial is the best reference I've read in the six years I've used D3. Previously, the documentation was too esoteric for the normal person to learn. Now, this really opens it up for more people. Nice work.

D3 is great, but last time I looked at D3 it worked a lot more like jquery than modern frameworks. Is it still worth using or in the decade there are better ways to work with React/Vue/Angular.

One of the primary goals of this new introduction to demonstrate that D3 is not primarily a DOM framework, so you can use it with React, Vue, or Angular if that’s what you prefer. I encourage you to read the post!

Thanks! I feel honored you replied. I'm mostly a backend programmer who tries a bit of front end work so dont have a lot of time to learn everything. I will try again because every beautiful visualization I've seen is d3 based.

There are a few libraries that wrap D3 to bring it to react (recharts, vx, nivo, victory), but my experience is that to program in the "react way", you need to push chart updates through React's state management system, which is slow. If you need a chart to be really responsive, you have to manage the callbacks and dom manipulation yourself, and then you have the dom getting out of sync with React's concept of it, and it's all messy and broken. Haven't found a good fix for this yet.

Had decent amount of success with learning plain SVG and generating it from Angular/React/Vue.js. It's very nice to be able to mix some SVG into your webapp, lot's of tools can generate SVG for you too...in one of my home projects I drew my apartment in Draw.io, exported it to SVG, copied it into a React project and added some lamp icons. Now I have a custom IOT frontend for my ikea lamps.

I learned it a little bit when working on my scientific analysis tool for single cell RNA-seq analysis. It takes time to learn, but there are many examples out there.

https://alona.panglaodb.se/results.html?job=uOBMsjyQfFK8G7wQ...

I'll provide another voice to support the idea that d3 is a really great library. For anyone who has tried to learn it in the past and found it a bit too complicated, I'd suggest you approach it with the following perspective: Learning d3 is more about learning SVG than it is about learning d3 (unless of course you already know SVG).

There are more and more articles on hacker news where I find myself reading an article about X, only to find myself wondering the same thing as before I read the article: "What is X?".

D3 has fascinated me for the past 3 years. I've spent this time embedding interactive charts in mobile apps. Thanks for creating this

Interesting. Is there a reason someone would prefer D3 versus say R's ggplot2 or Python's seaborn for a non-interactive graph?

For graphs beyond ggplot2, I am happy for the various D3 to R packages, such as

networkD3 (for Sankey diagrams) - https://christophergandrud.github.io/networkD3/ ,

R2D3 - https://rstudio.github.io/r2d3/ , and

htmlwidgets - http://www.htmlwidgets.org/

One of the reasons I love d3 is that it's built on standards such as CSS and SVG/HTML. They are so widespread you can't go wrong learning them, and that knowledge is immediately usable elsewhere (other related benefits include extremely powerful graphic capabilities and tons of documentation).

Reply on news.ycombinator.com