Why xkcd-style graphs are important | Hacker News Reader

Why xkcd-style graphs are important

chrisstucchio.com

296 points001sky75 comments

Readers weren't confused by the overly precise graphs, we were confused by the entire premise of the macroeconomic Monte Carlo article.

And they weren't angered because of misconstrued intent, but because it was stated that anyone who supports basic income "borders on innumeracy", and the claim was backed up by a toy simulation that addressed roughly none of the potential effects of that policy.

The article also attempted to reduce the concept of policy debate into a STEMlord "show me the code or you're wrong" absurdity.

Obviously though, the author has learned his lesson. His lines weren't squiggly enough.

I looked at the example graphs and thought, what the heck was he trying to prove plotting income and job. Thanks for clarifying.

I didn't have an opinion on basic income before (though a while ago I came across http://philip.greenspun.com/politics/welfare-reform.html, which seemed interesting), and I didn't come away from the article and comment thread with an opinion on basic income, but I was very pleased to see a proposal for modeling ideas with computation.

It's also very nice to see an adjustment on that proposal with some new information. It wouldn't have occurred to me that making a chart look hand-drawn would be effective in implying less precision, but now I have a new idea I can try out for graphics demonstrating rough ideas.

I think Chris's tone can be blunt and read as unaccommodating, which may be distracting from the his points, but I think I prefer it to the overly casual tone many adopt when stating an idea. That tone isn't great for customer service but it's very helpful when trying to understand an idea and whether or not to accept it.

Also, it is important to base comments in fact. He accused people on both sides of innumeracy: "Like most political arguments, the discussion of a Basic Income borders on innumeracy. So I’m going to take the opportunity to launch into a far more interesting mathematical tangent, and illustrate how to use Monte Carlo simulation to understand uncertainty and make business/policy decisions. I promise that this post will be far more interesting to python geeks than to policy wonks. I’m just picking Basic Income as a topic to discuss since it showed up on hacker news yesterday and it’s a topic where I see basically no numbers whatsoever."

Unfortunately he was unwilling to reply to my proposed NOPTransferPayment, which is a system in which each person pays himself or herself a transfer payment. If the payment is large, his preferred model indicates that the cost of the policy is very high, but if the payment is negative, then his preferred model indicates that the policy creates a very great amount of value out of thin air.

It is remarkable that someone would stick to such an obviously broken way of modeling the costs and benefits of transfer-payment-like policies (or of policies which have large transfer-payment-like parts, like Basic Job). He even goes out of his way to call attention to his continued insistence on this method months after the fact.

Some people understand subject matters just enough to make a statement but not well enough to analyze others' statements. Economics in particular seems to be full of these people

His other posts lead me to believe he he is more capable than that.

Maybe he didn't see it? No one else responded to the comment either: https://news.ycombinator.com/item?id=6727029

It's hard to take many interesting ideas out of this implication of bad faith on Chris's part. Wouldn't it be better to talk about the topic of the post?

I hoped that he would see it when it was cross-posted on his blog's comment section, but I also agree that this tangent is not particularly useful.

I completely glossed over your comment because I thought it was just a snarky post advocating no transfers. Clearly I misinterpreted it, so I'll take a closer look.

From this comment it sounds along the lines of classic "man marries his maid" macro critiques.

The only thing I strongly advocate is actually thinking things through numerically rather than tribally. If you have some alternate accounting scheme I'd love to hear it. The only thing I object strongly to is the anti-rationalist "accounting is impossible, yay/nay basic income" types.

Great :)

> From this comment it sounds along the lines of classic "man marries his maid" macro critiques.

I'm actually not very well versed in this area, so I had to look this up. You're correct, the basic mechanism of the critique is identical.

There have been a lot of scathing critiques of your position, so I'll try not to repeat them, especially since I basically agree.

I'll put more thought into this, but coming up with a numerical justification for favoring one policy over another, without real-world data, seems really hard. If you take out the transfer-payment-like portion of both policies (nearly all of Basic Income and most of Basic Job), you can see that Basic Job has purchased something (jobs that need doing, but wouldn't pay well) and Basic Income has not. At this point, Basic Job has done pretty well.

The disincentive to work when receiving a stipend that is sufficient to live on is definitely real. However, it's a bit hard to address the benefits associated with increased physical mobility and entrepreneurship under BI. Apparently some studies observe a decrease in economic activity upon the introduction of BI and others observe a decrease[0]. (Interestingly, some of the situations described are shades of "man marries his maid", like the mothers who stop working to take raise their children).

The important thing to me is that I don't think there's much use in comparing the balance-transfer-like parts of the programs. We should compare the outcomes, because the fact that one program taxes and pays a stipend to everyone who already has a job and the other does not isn't a huge difference on net. I was a bit too snarky in the past, but this is the portion of the policy that is NOPTransferPayment-like. I'll go a bit further and claim that programs which arbitrarily redistribute income don't have a big economic effect; Social Security for instance does not greatly increase or greatly decrease the level of economic activity (assuming that it just pipes a fixed amount of income from young people to old people, which it doesn't, but anyway...).

* ahem *

Now that I've gotten myself to write this all down, I realize that my objection is actually compatible with your model, and I've just made the error of thinking otherwise because the dominant factors in your analysis were the ones I wish to disregard. I have installed the required packages, made some tweaks, and run your simulation. If we do not count money taxed and handed right back to citizens as a cost, the difference is small enough that reasonable people could disagree about which program to prefer:

http://pastie.org/8987586#

http://i.imgur.com/UeWTSko.png

This quick and dirty run is biased in favor of BI - we've accounted for the disincentive to work at all due to BI, but not for whatever disincentive to economic activity introduced by setting taxes higher than under BJ. This wasn't a problem in the original analysis because it regarded the whole taxed amount as a cost. This factor should be some fraction of the difference of the amounts of tax between the two programs, but it is not clear what fraction. In fact, I've found a source that seems to indicate it should be a fraction greater than 1, in which case my whole objection could just make BJ win by an even larger margin.

Perhaps I should not say this, because I have not modeled it... Here we go anyway: It seems very likely to me that both of these systems are better than our current system of somewhat excessive overhead and near-100% marginal tax rates for the non-working poor.

[0]: http://en.wikipedia.org/wiki/Basic_income#Disincentive_to_wo...

(Interestingly, some of the situations described are shades of "man marries his maid", like the mothers who stop working to take raise their children).

That's not really the same thing. The point of "man marries his maid" is that after the marriage GDP goes down in spite of an increase in economic activity - the maid continues cleaning his house, but also cleans his pipes.

Women staying home for family work is a real change in economic activity. Domestic labor may not be properly accounted for, but it's not a no-op as with the man marrying his maid.

In any case, merely tracking production and ignoring the paper cost of transfers is a fairly valid way of looking at things. I don't fully agree with it, because I do consider it a cost when a productive worker loses some of his productive capacity, even if another person gains an equivalent amount. So I certainly do want some penalty on the actual amount transferred - but you are certainly right that this should be sum(tax(i) - transfers_to(i)), not sum(tax(i)) (where i represents a given person).

I'm definitely feeling like an idiot, ignoring one of the most interesting comments on post. You've definitely given me something to think about.

Very well said, and thank you for taking on the task of debunking this flawed analysis. It is a mostly thankless task since the people who would be on your side (e.g. people with a good understanding of economics) tend to avoid articles like that in the first place.

> but because it was stated that anyone who supports basic income "borders on innumeracy"

This is a blatant misrepresentation of what he said. The exact quote is: "Like most political arguments, the discussion of a Basic Income borders on innumeracy."

First of all, it says nothing about the supporters of a basic income being innumerate, but that the discussion itself (on both sides) is. If you bother to actually _read_ that paragraph, his point is that discussions about basic income don't include an attempt to make things concrete with hard numbers. It's a fair complaint that his analysis is only a baby-step in that direction, but he also explicitly addresses that, saying that he was just giving an example of _how_ one would go about bringing more numbers into the debate (i.e. the missed potential effects that you noticed).

Ah, I remembered that bit incorrectly. I should have gone back and read the article again in full. His having taken sides on that issue colored how I read and remembered the article.

I definitely did not walk away from reading it the first time with the impression that it was a simple illustrative example of how one might bring evidence into policy debate and how simulation can provide evidence when other forms aren't available. I'll try a re-read with that in mind, as I would have quite enjoyed that conclusion. Perhaps opponents to BI were able to get to that bit undeterred.

It might be that unassuming squiggly imprecise graphs might be only the beginning of the required revisions if that was to be the core takeaway.

I'm pretty strongly in support of a basic income (or at least the exploration of the idea), but it's possible that the version I read was after some updates to the article that made his intent more clear (e.g. the PS and PPS appear to have been added after the initial publication).

It's not even about bringing evidence to the debate. I had planned on doing that in a followup blog post about Markov Chain Monte Carlo, but I haven't gotten around to finishing that one yet. This post was simply about clearly and quantitatively expressing your beliefs.

Numbers make people feel intimidated, and threaten their ego with the potential that they might turn out to be wrong. That's why most people avoid them in political arguments.

This is another great example of an attempt to bring numbers into a political argument, this time about sustainable energy: http://www.withouthotair.com

Numbers make people feel intimidated, and threaten their ego

I'd argue its stronger the other way around: people in political arguments falsely insert (crap) data to intimidate other people and polish their own ego.

I didn't see that article the first time around, but I have to agree with the parent post; when I read today's post, I saw the graphs and didn't understand what they were trying to say (with or without xkcd-ification). So I quickly skimmed the December article for context on those graphs, and I still didn't understand what they were actually representing.

Rereading that section, and the code, I slowly decipher it... they are presenting histograms... I see labels "cost_benefit", so presumably that's a ratio... oh wait in the next paragraph he writes "cost - benefits" so I guess it's a scalar... but I still don't see why the side-by-side histograms are presented with varying-width buckets... oh, because he fixed the Y-axis, which constrains the bucket widths....

The problem, as lotyrin implies, isn't that the "lines weren't squiggly enough". It's that it's a graph that's hard to read and understand what it's trying to say, due to a total lack of labels or context.

(I also take issue with a lot of his assumptions and even his definition of Basic Income, but if we take in good faith his claim that his Monte Carlo article wasn't about economics and public policy but about python and using the scipy library, those are not necessarily complaints about the article.)

This. This might seem like a minor nitpick, but plots without axis labels are a big no-no and far from 'publication quality'. In academic publishing the general editorial goal is to aim to make your graphs as readable as possible on their own, without forcing the reader to refer to the article to try make sense of them. Just my two cents.

This reminds me of my favorite Java project, designed to make work-in-progress programs look exactly that to non-technical stakeholders who otherwise assume 90% complete UI means 90% complete application.

http://napkinlaf.sourceforge.net

From that same page is the essential "Don't Make the Demo Look Done" post by Kathy Sierra:

http://headrush.typepad.com/creating_passionate_users/2006/1...

It drives at the same point as the original author, just from a different angle.

In a related vein, the idea reminds me of a strain of computer graphics research known as "non-photorealistic rendering"[0], i.e. acknowledging that for some uses, building a full shading model and trying to mimic reality actually makes the image worse rather than better. Once you've made that realisation, you can design renderers that have flat shading or heavier outlines or any number of other features that make it look more "diagrammy". I guess that xkcd graphs (and napkinlaf) are arguably themselves instances of NPR.

https://en.wikipedia.org/wiki/Non-photorealistic_rendering

This is also a problem in architecture. When a photorealistic render of a building is presented to a client the client assumes it will be built just like the render (beautiful sky and green trees included).

Whereas the public, upon being presented with a photorealistic render of a building, immediately go "sh, right, as if" and start imagining the car parks and advertising hoardings and ...

(cf the recent renders of "London's Garden Bridge" which were gently mocked for their "optimistic" view)

> i.e. acknowledging that for some uses, building a full shading model and trying to mimic reality actually makes the image worse rather than better.

Think all 3D computer games before, like, 2010. Part of me wishes they made this realization; the other reminds me that we wouldn't be where we are today without a decade of crappy-looking games.

Webdevs need a napkin bootstrap theme for their MVP. Wondering how hard it would be. One would have to use CSS3 transform I guess.

I use Balsamiq for this and love it.

http://balsamiq.com/products/mockups/

I like that - akin to a live-action Balsamiq.

One thing you may want to try, if you're not doing this already, is to remove the actual numbers on the axes when you xkcd-ify graphs. I think people see the numbers and think that the results are precise and removing numbers that mean very little can have a pretty big effect on people's perception of the precision of the results. I say this because your first example still has the numbers on the axis ticks.

Agree.

And if the numbers still feel important, don't put more than one or two of them. Knowing that the axis goes from 0 to 10k is probably enough information for something deliberately imprecise.

I came here to say exactly this - the purposeful imprecision in plots doesn't come from squiggly lines (which I consider unprofessional in many contexts), it comes from the lack of axes numbers. That first xkcd plot is the worst possible combination of communicating false precision while looking silly.

Context is also important. Sometimes, your audience will want to know exact numbers and imprecise plots will look bad. Imprecise plots should only be used to explicitly show a trend, which your audience should be expecting. If this is the context, they'll understand and squiggly lines are unnecessary.

I disagree with this. It drove me nuts when my first-year economics professor had a perfect Excel graph containing two crossed straight lines to demonstrate supply and demand. None of it made sense to me until she showed me a graph from her third-year course with actual data in it. xkcd-style graphs would have helped me with my understanding, as would drawing them on the blackboard.

As a potential counterxample for when you might want to remove numbers, a couple weeks ago I was presenting a proposed analysis pipeline for some data, and what I'd expect the results to look like. The x-axis points were all fixed, and clearly tied in to numbers I had discussed previously, and the y axis had experimentally meaningful ticks at 0, 1/2, 1, and 2.

I was, in fact, told later that the xkcd style graph was extremely helpful in conveying that it was not a graph of real data yet.

This guy links to: http://matplotlib.org/xkcd/examples/showcase/xkcd.html

I've been in the matplotlib site a few times before and I noticed that the logo was weird. Turns out the the matplotlib website has a mode where all examples are rendered with xkcd, all text is converted to Comic Sans and all other sort of funny things happen. You just have to add the xkcd keyword in the url.

Here's another example of a documentation page completely unrelated to xkcd.

Original Style: http://matplotlib.org/examples/lines_bars_and_markers/line_d...

xkcd style: http://matplotlib.org/xkcd/examples/lines_bars_and_markers/l...

Those graphs look hideous. The wobbles are too big, and too regular.

Compare with an actual xkcd graph: https://xkcd.com/1306/

counterpoint http://xkcd.com/1064/

And I think you're missing the point of the post...

I think a fair compromise would be to have a setting to adjust the 'wobble'. I like both charts, but it mostly looks like Camillo prefers the font (only speculating).

The regularity of the wobbles makes me think there's something wrong with my monitor. Very unsettling.

The author is conflating two very different issues:

1.) Imputing the imprecision of a mathematical model through stylized graphs.

2.) Choosing a firestorm of a political topic to demonstrate mathematical modeling.

A long time ago (by internet standards), Joel talked about that too in the context of software development, how appearance convey some degree of completion to people who can't see inside the source code. This led to this: http://napkinlaf.sourceforge.net/

Ironically, I trust Randall's graphs more than any professional-looking graphic in a newspaper or magazine.

Agreed... the irony of all of the xkcd-style plots is that Randall is usually very precise with his figures.

I think instead of error bars, one could use blur to describe error http://blog.velir.com/uncertainty/index.html

to me that kind of insinuates likelyhood that the true value is closer to the middle, which isn't necessarily true.

1. If I want a confidence interval, I want to know the end points of the confidence interval. Trying to guess them from a plot like this is annoying and not helpful.

2. This doesn't work for bar graphs, lengths, etc. The uncertainty is often going to be symmetric around the point estimate, but your opacity forces an asymmetric representation of the uncertainty.

3. Box plots are great. If you want more detail than that, a thin vertical histogram or density is going to convey much more information than shading.

Like all tools you have to aware of its limitations, but I think shading can be a fantastic way to visualize uncertainty.

For example, the Bank of England occasionally uses it in plots of economic forecasts, where time is on the x-axis and things like GDP might be on the y-axis, eg. here http://www.bankofengland.co.uk/publications/Documents/inflat.... The fading out of intensity over time is a great visual reminder that predicting the future is hard.

It is much better when your chart is supposed to be targeted at the general public, because the "smearing out" of the data is very hard to misunderstand, unlike confidence intervals.

From the first few graphs I've seen in the links, the shading is discrete and not continuous (e.g. p 40). Discrete shading does address my first point, and it can help somewhat with communication too. Personally, I prefer simulating from the implicit model and plotting, say, 1000 hypothetical sample paths; but I agree that discrete shading can be effective too.

I love it! I heard about this years ago and I think it's a great way to fuzz 'the science' and make it easier to think about. Look at it like this, as a 10 year old, you look at these graphs and see someone that can draw just like you. If you look at a graph of straight lines and precise blues and boxes and stuff, you get discouraged and think you'll have to work really hard for a long time to get to making computers do that.

Also this comic is somewhat related too: https://xkcd.com/1133/ If you can't figure out a way to relate your ideas in those 10 hundred words, you aren't thinking clearly enough. To help, here is the list of words: http://splasho.com/upgoer5/phpspellcheck/dictionaries/1000.d... and here is a checker for you to proof against: http://splasho.com/upgoer5/

I'll admit to having a similar strategy for mechanical drawings. I once discovered by accident that if I drew a pencil sketch, the machinist would understand that I wasn't asking for super precision, and I'd get a satisfactory part quickly. Or they would call me and ask: "Can I make it this way instead" or even "what are you really trying to do?"

Pretty sure that's why Balsamiq mockups look they way they do. Same principle.

I love this use case. This is also why I always make wireframes in black and white, and also keep them chunky. This automatically lets the viewer know that it's not a final mockup, and that they should comment on the high level UX rather than nitpicking on pixels.

I totally agree. This is the way most of economics is taught; concrete numbers provide evidence for ideas/models, but when learning their concepts, they simply slow you down.

Mathematicians are also taught to think this way, as is everyone else. Humans don't think in terms of numbers.

I don't know why you were downvoted; it's the same thing in CS. Why else would we reduce all sorts of algorithms to simpler big-O notation other than to convey the more important concepts of complexity tradeoffs in algorithms? Also, Fermi estimation (always relevant: http://what-if.xkcd.com/84/)

Those graphs he converted into xkcd style are bad examples. Almost all the xkcd graphs I remember reading are illustrations for simple concepts. Well, let me put it this way. The graph is usually compact, small and a few simple annotation/caption attached to the graph. That's exactly why the xkcd example in the article works well. Try to scale this up to a large graph like the one author did. The curly lines just hurt my eyes.

I like the general direction of this thought. Some things are hard to communicate from a cold start. You need to already be thinking in certain terms in order to understand. Probabilistic ideas and arguments are often like this.

An interesting example is science and scientific methods. Something starts as a theory (eating fat makes you fat) and accumulates evidence for and against it. Evidence "against" the theory doesn't necessarily outright disprove it. It just weakens the claim in different ways. It can lower the probability that it is true (maybe it just seems like eating fat makes you fat, but it really doesn't). Sometimes the magnitude is smaller (eating fat makes you a little fatter, but not much). Sometimes it's kind of true, but the whole story is more complicated and the statement in the their theory needs to be made more precise (increasing fat consumption makes you fatter if everything else stays the same, but it also makes you feel full which makes you eat less of other things the effect only exists in controlled conditions).

The above example needs to be considered from multiple dimensions. People who haven't been thinking about this as a multidimensional thing have a hard time evaluating your statement in this way from a cold start. Experts used thinking the way they are explaining things think the public is stupid or uninterested. Even if they humbly agree that the public are just not experts, they still conclude the same thing. The public want everything boiled down to a simple statement that hides the texture.

X Causes Cancer

This KXCD style graph subtly conveys a little of that texture. I think this is a good thing. I really like XKCD. It's very good art for a very interesting definition of the word art.

I found myself conditioned to check for alt-text on all the xkcd-style graphs.

So authors: Another plus is that you can include a humorous caption for your graphs. I'm sure there's always a funny angle that you wish you could include but that wouldn't be an official part of the paper ...

I like this - it reminds me of advice when creating an early stage prototype: the higher the fidelity of the prototype, the more chance that your audience will think it is a final (hence set-in-stone) product.

I'd like to point out that there has also been some research done in order to see how to exploit these sorts of "sketchy" visualizations for information visualization. This paper by Wood et al. contains some very nice examples, as well as small user study:

http://openaccess.city.ac.uk/1274/

They show that users can judge the degree of "sketchiness" on an ordinal scale but that the judgement varies extremely between individuals.

I've done a Google Ngrams API that generates plots in XKCD style here:

https://github.com/econpy/google-ngrams

and a webapp version with slightly more advanced XKCD tweaks here:

http://xkcd.culturomics.org

Everything starts with matplotlibs xkcd lib but has been tweaked to produce plots based on the output of the Google Ngram Viewer.

I would have went with a squiggly line showing instead of a squiggly-histogram for full effect, but otherwise I think this is genius. Showing people information which is "correct" for all ways that information may be consumed is a critical skill these days, you cannot blame the reader and still be an effective communicator.

If you're looking for a UI mockup tool with support for back-of-the-napkin style drawings, I'd recommend the Pencil Project [0]. It's easy to use, open source and available on every platform and as a Firefox extension.

[0]: http://pencil.evolus.vn/

The biggest difference I see between the original plot and the xkcd() one is labels! In your article, there are no labels on the axes. Thus the numbers are context-less -- providing labels (and perhaps adding text at particularly important points) is more important the actual style.

Hand draw them using something like Paper by 53 on an iPad, or even in Paint...then they have the right essence of looking hand drawn (?messy) and people don't ask the question. And tbh, probably take less time as well.

I would be an interesting challenge to automatically generate XKCD graphs from real data (simplify, exaggerate, render as hand-drawn etc) in the same spirit as non-photorealistic rendering.

That's how the graphs in the article were generated. A bunch of people played with it here:

http://stackoverflow.com/questions/12675147/how-can-we-make-...

I must say that this post was quite convincing. However, I wonder if that really is Max Planck in the comments. If so, then I'm far more inclined to heed advice from him.

I think the graph has opposite effects to avid xkcd fanboys.

Balsamiq is great for this.

April fools?

Not a fan.

There is already a way to communicate imprecision of results.

Error bars

Got it?

Nope. Error bars are precise.

True. The original plot claims to know one thing (the reported quantity). The plot with error bars claims to know two (the reported quantity, and the correct error bars).

Often getting real error bars is much harder than getting the original quantity.

Reply on news.ycombinator.com