D3.js 5.0 is out | Hacker News Reader

D3.js 5.0 is out

github.com

407 pointstolien74 comments

Wow, didn't even know about the Fetch API. XMLHTTPRequest, the API that changed software forever by igniting a wave of more powerful browser applications, is finally going to be replaced. Funny that the last 20 years of the web were all built on this ridiculous API that Microsoft came up with to try and make Outlook work in a browser. Good riddance, but you will be missed. Not that any of us has used that disgusting API directly in the last 10 years.

>Not that any of us has used that disgusting API directly in the last 10 years.

Well, duh. It was a lower level API, offering more control.

Fetch isn't much better. Half-arsed at best, and also needs a higher level wrapper to achieve the same terseness as popular AJAX utility libs.

> Fetch isn't much better. Half-arsed at best, and also needs a higher level wrapper to achieve the same terseness as popular AJAX utility libs.

Really?

   let response = await fetch('http://www.example.com');
   let json = await response.json();

It's not not terse.

That's just for a GET.

Sending e.g. a JSON POST request requires you to specify the content type and to manually stingify the payload.

It's better than XMLHttpRequest but not as terse as the tons of helper libraries that we used (instead of XMLHttpRequest) anyway...

It's been around for 2 years, I noticed it when github released a polyfill.

It's been great to rely on a standard function, than any of the libraries to make http requests.

I still use on axios over fetch. Mainly because fetch doesn't have abort function. axios implements cancelable promise proposal that didn't go though TC39

Tho it has been a while since I checked TC39 status on fetch

I also choose Axios over Fetch for now. I had the following post that made the case - https://medium.com/@thejasonfile/fetch-vs-axios-js-for-makin...

Thoughts?

Fetch requests do support cancelation for a while now through `AbortController`. See: https://developers.google.com/web/updates/2017/09/abortable-...

Which doesn't work even in Chrome.

Fetch is a Web API, not an ECMAScript language feature, so TC39 doesn’t have much to do with it. They do do Promises though and Cancellation is a Stage 1 proposal [0], so it’s more than a year away from standardization at best.

WHATWG handles Fetch and their spec[1] was updated seven days ago.

[0] https://github.com/tc39/proposals/blob/master/README.md [1] https://fetch.spec.whatwg.org

We used to do that stuff using frames back in the day. You’d have a hidden frame, change its url and look at what that dom contains. At scale, too. I was amazed at all the hoo-haa around that “new” xmlhttp API

We did the same thing, and to dynamically pull in content into the parent context, we created our own script block tag. <div type="javascript">...</div> so that we could extract the JS and eval it in the parent context. The clunky thing was that you had to hand escape reserved html characters (&, <, etc) in your JS code. So many bugs...

Wow, then I think that I jumped directly to that hoo-haa since I didn't even consider the technique you are describing, I wasn't expecting today learning something from dynamic HTML past of the web. Thanks!

Yea there were a few things like that to emulate it. Things like JSONP + document.write() were also very common, though very ugly.

Fetch API still cannot handle progress events, unfortunately, so it is not yet a backwards compatible replacement.

And until recently it could abort() either, but Chrome/FF/Edge all have it now (yay!) and there is a polyfill for Safari/MSIE and old browsers: https://www.npmjs.com/package/abortcontroller-polyfill

Shout out to Philippe Rivière for the awesome work on d3-geo-polygon[1] (mentioned in the release notes), which enables tons of new map projections like these[2].

Philippe, if you're reading this, thank you. I'm really excited about d3-geo-polygon and I know a lot of others in the community are too.

[1]: https://github.com/d3/d3-geo-polygon

[2]: https://beta.observablehq.com/@fil/polyhedral-projections-wi...

What are the use cases for this functionality? Genuinely curious.

Unusual map projections like these are often curiosities, but the can also be useful for certain topics, like mapping migrations[1][2][3] or showing airline routes[4][5].

[1]: https://commons.wikimedia.org/wiki/File:Map-of-human-migrati...

[2]: https://nordpil.com/static/images/redknot_migration.png

[3]: http://ngm.nationalgeographic.com/ngm/0603/feature2/map.html

[4]: https://imgur.com/5TFPcDa

[5]: https://imgur.com/6X91323

Additionally, this project is a generalization of a feature that already existed in D3 (clipping spherical polygons to map boundaries -- previously you could only do this if the boundary in question was the antimeridian or a small circle centered at the origin). That generalization will make it easier to use some more common map projections that were already supported in d3-geo-projection[5] like as Goode's interrupted homolosine and Mollweide projections. These are useful projections for showing things like population density[6] where you care about preventing distortion in the landmasses but don't care about the oceans. While these projections were possible before, they required the use of SVG clip-paths and other tricks to prevent drawing outside the map bounds. In theory they could now be simplified using the polygon clipping implemented in d3-geo-polygon.

[5]: https://github.com/d3/d3-geo-projection

[6]: http://www.jakelow.com/essays/mapping-the-world-population

Awesome. Thanks for the informative reply!

The sad state of affairs is that while D3 is brilliant in terms of API design, it is fairly low level, so few people use it directly. I've been using tools based on D3.js (like C3, NVD3) and it's been a joy.

However, last time I checked (~mid 2017), most of these hadn't migrated to D3 v4, so I'm not sure how fast v5 adoption will be. Maybe some will skip v4 altogether and go directly to v5?

At this point, I'm not sure which wrapper to use, because I don't want to get stuck with v3.

You could change that.

I loved using D3 directly and found it amazing.

I've been using Billboard.js as a drop-in replacement for C3. It's a fork made specifically to work on D3 v4+.

https://naver.github.io/billboard.js/

Amazing, thanks. C3.js has been a real pleasure to use, so I'm glad this fork exists. Cheers.

For the people who don't use it directly, give vega and/or vega-lite a try:

https://vega.github.io/vega/

https://vega.github.io/vega-lite/

Low level, yes. But I happen to like it as an alternative to jquery.

I needed a custom stacked timeline for the work system.

I was planning to do something horrible and then remembered d3 from a post years ago, day of futzing around with examples and it does it cleanly in less than 300 lines of code.

Genuinely impressive API as well.

Finally, `d3.create`[1] - it has been a minor pain pointer in d3 to create a detached selection for quite some time :)

[1] https://github.com/d3/d3-selection/blob/master/README.md#cre...

I'm looking to get my team into using D3 more as we begin to do more graphing and data visualization. Do any of you have recommendations on tutorials, classes and other learning aids?

Promises instead of archaic callbacks. That's a feature I'm excited about!

promise is callback. When used in the context of the simple data loading scenario typical of D3, there is no difference.

Not exactly and not necessarily. You can await a Promise and not have any functions at all:

    await thing()
    console.log(‘Thing done’)

Also you could attach multiple callbacks to the same Promise at any point, even long after the promise fulfilled.

Promises compose, bubble errors, lift sync errors, can be awaited, have combinators like Promise.all(), they're where the ecosystem headed, and more.

Thinking that promises are callbacks because .then() takes a function suggests misunderstanding.

They aren't even executed in the same queue.

That seems like a small update compared to v4. Wouldn't it be better to brand it as a minor version and provide backwards compatibility with deprecation warning where Promises were added?

If following semver style: Major updates include backwards incompatible changes. Minor updates include new functionality but no backwards incompatible changes. Patch updates include no new functionality.

There are backwards incompatible changes, so it has to be a major bump.

OP was suggesting not to have included backwards-incompatible changes, using deprecation warnings instead and branding it as a minor upgrade

Hence OP was suggesting not following the semantic versioning model.

Why would they do that? To have extra work and to have to carry deprecated garbage and compatibility ports for a few releases more, instead of making a clean new version?

Those that need compatibility can stick with 4 until they're ready.

The semantic versioning model just says how you should update your version if you break compatibility, not that you can't avoid breaking compatibility.

In fact, they suggest the opposite: "Before you completely remove the functionality in a new major release there should be at least one minor release that contains the deprecation so that users can smoothly transition to the new API."

https://semver.org/#how-should-i-handle-deprecating-function...

Either my memory fails me, or the comment was edited...

I don't recall responding to a comment that mentioned using deprecation warnings instead. :/

Sorry, I edited it right after posting it. Didn't think somebody could answer that fast :p

Regularly occurring major versions, with only a small amount of changes, has actually kind of become seen as best practice in the JavaScript community. React, Node, Angular and a ton of other popular tools now release a major version update every year or 6 months. The reasoning is that by taking small, backward-incompatible steps, then developers are more likely to make the effort to install the latest update.

To understand the problems the community is trying to avoid, one only needs to go back to 2015-2016, when Angular 2 was released. At the time, AngularJS (aka Angular 1) was extremely hot; it had in record time become THE framework for any front-end developer to learn, and it looked set to dominate the landscape for years. And then, Angular 2 came out and it had a TON of changes. The learning curve of some elements of Angular 2 was very high for the average developer (TypeScript, reactive programming). And worst of all, the "migration path" from AngularJS to Angular 2 essentially amounted to a rewrite of your front-end application. Most never bothered updating their app, and most devs moved away from Angular, especially when React was released. All of that AngularJS experience that we accumulated became "outdated" in the span of about a year.

Sorry, but this is a rewriting of history and the present facts, and I can't let it go unchallenged.

React was first released in 2013, whereas Angular (2) was first released in 2016. React was already a very popular and rapidly growing framework by that point, so the idea that people adopted it simply because of Angular 2 is clearly false.

And while the migration process from Angular 1 to 2 was painful, it's simply untrue to say that nobody bothered, or that every AngularJS developer moved away. If you look at a source like https://hotframeworks.com/, which sources framework popularity data from GitHub and Stack Overflow, you can see that Angular has achieved a significant amount of adoption, and there are plenty of examples of people migrating successfully from 1 to 2+.

Sorry but your facts are just wrong. Angular2 was announced in late 2014, just about the time when angular 1 was becoming mainstream used by big banks and the like. Look at Microsoft's blog from early 2015 announcing the use of typescript in Angular2. Linking some generic website does not make your comment less toxic and I would strongly suggest you get your facts right before posting stuff like that here.

Semver is awesome. If the owner adheres to it, then it can be a very strong signal to the downstream users about whether they should be able to [probably] upgrade seamlessly vs possibly having to do some major rework. Also, if you see an adherent of semver bumping the major number all the time, you can choose whether you want to sign up for the churn on subsequent releases or not. Semver seems to encourage backwards compatibility. Not force, but encourage. There's a reason Microsoft and Intel worship backwards compatibility. Your downstream users love backwards compatibility.

> D3 now uses Promises

Thank the lord!

I'd suggest thanking the developers instead. :)

You are right. Much thanks to mbostock, curran, micahstubbs, jefkoslowski, gordonwoodhull, and baerrach.

Its funny, when I landed my first job as a software developer, in a statistical startup, I learned about promises using d3.queue[1]. This was before javascript had native promises, and around the javascript community was settling on A+ promises. But d3 was how I learned to write asynchronous code without relying solely on callbacks.

[1]: https://github.com/d3/d3-queue

Likewise, but in jQuery, which in one version introduced a feature where you could add listeners to an ajax request and it'd get called after the response returned - I was like what? how? whoa!?

sarcasm? who cares about D3's data loading APIs... i mean most sophisticated JS apps using D3 wouldn't use D3 for data fetching from the service.

https://github.com/d3/d3-queue has 1360 stars and gets 50% as many daily downloads as d3 itself, so I wouldn't be too sure that nobody out there is using d3's async facilities.

Any good introductory tutorials for D3.js?

Mike Bostock (creator of D3) has a bunch of great code examples on bl.ocks.org.

I would start with his General Update Pattern (the link below goes through 1-3), which is key to understanding D3 as binding data to SVG elements (hence the name Data-Driven Documents) and animating any transitions.

General Update Pattern: https://bl.ocks.org/mbostock/3808218

Also the documentation is great, so check out the intro here:

https://d3js.org/#introduction

btw, if you're using D3 purely for charting, expect a lot of work to get something that looks decent. There charting libraries that have taken care of the styling and add functions on top of D3 to make your life easier.

Thanks! Very useful.

Are there d3 only jobs? Aside from NYTimes, who is using d3? Thanks

We do at CB Insights (and we are hiring)

open for applications from europe as well?

We are using d3 a lot at our startup to build dashboards displaying huge amounts of data for our clients. We're just a small german startup though. But d3 has high adoption through the field of data vis.

i have the same feeling that D3 related jobs are extremely sparse around here in europe (NL/DE).

As niche consultant, we use a lot of D3+React for real-time dashboards (in the world of clinical informatics)

In addition to what et-al said, the d3 wiki [1] has a list of tutorials. If you're looking for a book, I found Scott Murray's "Interactive Data Visualization for the Web" [2] to be good.

[1] https://github.com/d3/d3/wiki/Tutorials [2] http://alignedleft.com/work/d3-book-2e

I've been self-learning d3 recently and I found Square's guide to be very helpful. It takes you through enough of the concepts that you're able to understand code snippets and understand what's going on

https://square.github.io/intro-to-d3/

Is the best way to accomplish sugiyama still to use dagre-d3?

sugiyama?

See [1], looks like a method for drawing digraphs.

[1]https://github.com/d3/d3/issues/349

As far as I know, yes. Or use the wasm version of graphviz.

dot is still the best. If you're okay with wasm, use Viz.js

http://viz-js.com/

Looks cool - would love to implement some things using D3! Would be interesting to know how people plan to handle browser incompatibility, or is it even a concern when approaching ~90% of browser usage support? [0] (for fetch, promises but mainly async/await in this case, if I understand it correctly)

[0] https://caniuse.com/#feat=async-functions

You'll have to manually include polyfills for both Promise and Fetch to support things like IE 11 or iOS 10.2.

Streaming responses is very cool

Incredible work, by an incredible programmer.

I love d3js, but I had the feeling it never got so widespread as jQuery. Similarly, from my feeling, it never had such a revolutionary impact. Maybe I am wrong because the data-driven approach (ie. data binding) inspired a whole generation of frameworks. But is there a showcase of famours sites using d3js for anything beyond visualizations (say in a similar manner how jQuery is used across all web page client logic)?

The entire purpose of D3 is to be used for visualisations, so no, it's not really used for anything like that and doesn't have as wide a scope as jQuery.

Reply on news.ycombinator.com