Banning exploration in my infovis class

Exploration is absolutely one of the key goals of data visualization. Tukey's insight was two-fold: First, that statistics puts too much emphasis on confirmation but not enough on systematic exploration of a data set. Too often researchers do not understand their data. They bring their own biases and preconceived notions, where they should be listening to the data.

Second, too often is visualization mistaken for confirmation. A curious pattern or an outlier may just be an artifact of the layout algorithm. The "find" part of visualization should happen through mathematical insight. Graphics can at best describe the underlying mathematical reality but no more. One cannot strictly speaking "find" anything, only form intuitions or illustrate already proven insights.

Perhaps the author's difficulty in evaluating his students work lies not in the exploration part of their assignments, but in his own pedagogic emphasis on "tools," "frameworks," and "users." None of those things are relevant to data visualization as such. They might be goals for a business built around data visualization (to produce tools, or to identify user needs). A university should offer more than job training. Those interested in users and in "what one gets paid for" would do better in an internship or at a more narrowly technical trade school.

I don't know how "finding" is any more of a goal for data visualization than "exploring." Data visualizations tell stories. They often support the first and the last step in data analysis: the exploratory phase and the presentation of findings. They are inherently subjective, evocative, concise, artful.

I've often made the mistake of emphasizing exploratory UIs in situations where the reality is that >95% of users are looking for specific solutions, rather than the chance of dicking around with the generic toolset that I personally found captivating.

It's really hard to step back from "I want to share this exhilarating exploration with everyone!" to "I'll narrow this down to a specific use case, and leave the rest of the potential for another day."

> In denying the student the ability to frame their main task as exploration, they are forced to concede that what they want to find is not what their end-user may be looking for and then: (a) engage with their client or “create” a reasonable one with real tasks and decisions, (b) understand the data and tasks much more deeply, and (c) identify good validation strategies (no more insights!).

> Maybe this is obvious, but when I started teaching I thought that being more open-ended about what I allowed was better. That somehow it would lead to more diverse, cool, weird, and novel projects. In some ways that’s true, but as I’ve argued elsewhere, teaching infovis is itself a wicked design problem.

Its not just infovis, I think this is a good teaching idea in a very broad sense. Reading other people's writing nowadays, people are apt to be very lazy with their language and this by consequence makes them lazy with their ideas. Infovis should ban "exploration." Architecture should ban "modern." Career centers doing resume reviews should definitely ban "utilize." I'm sure every field has such tropes that are maybe useful in the real world sometimes, but make for quite lazy school projects.

In fact I think he may be going a little overkill in justifying his ban on "exploration". He doesn't need to talk about weighing the pro/con here, if before he had a paucity and now he has a multiplicity of interesting student results, he's won and they've won. "Surprise" be damned. The kids can buy lotto tickets if they want to be surprised by big data.

For being so simple, restricting the common and obvious in classrooms is probably an underrated technique. This is widely done in photography classes, disallowing students from doing certain things, like making them only use film for a while, so they really have to frame photos and can't just snap 1000 and "discover" one good one, or restrict to annoyingly wide prime lenses on an assignment to take some good portraits, etc. These constraints, even though they are constraints, greatly reduce the samey-ness of results and make the students engage their brains.

a) Whether or not it's pretty and looks interesting will be enough for first year sales unless you have a particularly sophisticated buyer

b) If pretty and looks interesting are the only things it does, you'll get slammed at renewal time because no-one used it

c) You need to know what behaviour your users will change based on the tool. They may not know that yet. That's a pretty good sales pitch though.

d) If you're particularly cynical, find a way to give users a magical number they can change based on behaviour, that doesn't actually mean anything. cf: "Klout influencer score". The more opaque its calculation, the better. Add a slightly random element to how it's calculated so that users build superstitions about how it's calculated. Allow their boss to easily run reports on your users's magical score, and include rankings.

You're a teacher, not an employer. It's not your job to tell people what to be interested in. This is exactly the wrong attitude to bring to any scientific endeavor, but I guess you don't get tenure for letting your students dick around on their own.

All I can say is that heavens I didn't run into anyone like this when I was first learning technology.

However, in real-life, figuring out all of the specific ways in which a user might want to use your tool can be mind-bogglingly difficult. In many cases, the specific few use-cases works, which means you can often just automate away most of the infovis stuff and just get to the result.

But when you have more than just a handful of these use-cases, or the use-cases are not well enumerated, the generalized approach can work better, and that approach for infoviz is often "exploration". In fact, building the generalized tool is often a good way at discovering the more specific use-cases for later rethinking, simplification and capture-in-code.

In that sense, the exploration infoviz tool can act as kind of a meta-exploration tool for figuring out what your users really need when they aren't otherwise able to articulate it.

That said, I think he states in his article something that I see as a core didactic problem without identifying it as such:

>The student is often engaging in “exploration” for the purpose identifying patterns that influence their design. They are often missing background knowledge and develop it in this step. But this is not “exploration” for the analyst who may already have a mental model of interesting and uninteresting patterns.

So he is expecting students to make a tool suited for the mental model of an expert even though the students have no idea what that mental model should be (and without giving them any hints what that should be). If some motivated students try to derive those tools for themselves on-the-go, he'll permit that in a fit of generosity.

If the problem he has diagnosed is that the students don't know enough rigorous definitions and techniques to find patterns, maybe the curriculum should focus on teaching those instead of going a step further and asking them to build a visualisation tool based on that non-existent knowledge.

I work in a big pharma, and long ago I learned that our biologists and chemists had absolutely no patience for inquiry that lacked a basis in the purposeful exploration of mechanism of action (hypothesis). Without a guiding principle, the number of possible (and meaningless) patterns tends to explode combinatorically and there's not enough time in a dozen lifetimes to test all the nutty proposals your computer can generate in a microsecond.

Isn't that what the author is suggesting? If not, and exploration via random walk IS in fact a productive practice that's worthwhile, then what's the rub?

I wish I had asked myself "what do users want to find?" instead of "what do users want to explore?" the last time I built a dashboard. Perhaps people would have actually used it.

Exploration is a tool but giving people utility to actually use your visualization is more important. You should allow exploration but it shouldn't be the only thing that people will use your visualization for.

Link to the class I took https://www.evl.uic.edu/aej/424/