Explorable Explanations
explorableexplanations.com
explorableexplanations.com
I believe the effect of that one simple addition would amplify their effectiveness (or at least reach) by an order of magnitude.
After the massive work that their authors put in, the effort of one good read-through would be marginal. If the above surmise is true, you'd be in power law territory.
But you'd need a tool to make such an additional channel as trivial as doing one good read-through.
I'm a little surprised that Bret Victor doesn't do this for his pieces, considering his emphasis on the use of more human capabilities[0].
Implementing the read-back part is trivial [1], and available on all "modern" browsers.
Trickier is making the product that lets authors add this channel without manual tagging. But even this is becoming trivial with speech-to-text API's.
Anyway, I wholly agree with the OP that the first focus of documents in dynamic media should be improved exposition and, by extension argumentation.
[0] "The Humane Representation of Thought" https://vimeo.com/115154289
[1] For example, see a crude implementation at https://willshake.net/plays/Ado/1.1 (click a play arrow in the blue column.) The tracking code is at https://willshake.net/about/text_tracking
That said, there's also zero advanced tooling to create and edit synced audio, and nearly zero tooling for playback. I wrote my own for http://vitor.io/uxr101, available here: https://github.com/vitorio/hyperaudio-lite
First, consider the difference between two people reading aloud: one is new to the material, concepts, and associated vocabulary, and the other is the author. They are reading the same text, but the author's rendition will carry an understanding of the material that is absent in the other one—and not found in the text, either. Doesn't that difference comprise relevant information?
Shakespearean actors do this for a living: they stick to the words on the page, but they add another layer of signals to help unlock them for people. Technical and expository writing can benefit from this effect as well. Don Norman argues, for example, that thinking requires emotion, and the voice is an easy way to add an emotional connection.[0]
So I believe that any document, explorable or otherwise, is amplified by a human reading, when that human really understands and cares about the material. And again, for subjects that warrant these specialized explanations—which tend to require some specialized language—the value added is particularly great, because it teaches people how to read something new.
Second, I strongly believe in an "all-in" approach for things that you care about. Clearly these authors care about reaching people, and I think there can be no doubt that the ability to click "play" brings in listeners who would never have been readers. And by giving a passive "flow" to the presentation, it helps sustain many readers who would have given up.
Adding a channel does not "take away" anything. Everything, including the interactivity and the opportunity for non-linear reading, is still there. (And note that the recording doesn't have to be linear, anyway.) The authors are not competing with themselves, they're competing with YouTube and notifications and other low-friction distractions.
So I'm excited about the OP. I would like this community—mature, passionate thinkers and teachers—to be the avant garde of future media.
[0] Emotional Design, Don Norman
Not to me, as someone who's never run into the material before. There's no knowledge-carrying particle that's transmitted vocally.
In fact, to me it would be a distraction. I learn better visually; I can stop reading, and start again, and quickly re-scan what I'd read before to catch up. It's harder to do that with a recording of a voice, and takes longer.
Other people might say the same about the graphics, or the interactions.
The point is that it's not zero-sum—no one loses out by the presence of other channels, if the document is dynamic. But if some people do benefit, then there's a value proposition.
That being said, I don't know how a spoken narrative would integrate into a piece of text. You would likely not want someone to read through the text as written because a spoken explanation will generally differ in phrase structure (shorter, less verbose) than written text.
Consider how much time it took you to learn Gaussian elimination from a book or lecture vs how long it would take to see an animated example and then pick up the details. Or consider how ambiguous mathematic notation can be [1].
So much of mathematics (purportedly the most structured knowledge produced by humankind) sleeps in mountains of unread journals and inside the brains (software) of mathematicians that are trained to do these visualizations on wetware.
It is a feature of math that these aren't hard-coded, as in the abstract one can conjure up any edits the imagination desires without the complex implementation details typically required by a computer. And in some cases a visualization (such as geometry) can be misleading and reduce generality [2].
However, there could be much to be gained by the majority from the hypothetical platform. Computation packages like mathematica only incidentally help visualize, as a byproduct.
Take for instance the videos by 3blue1brown [3]. Their appeal (and those in the parent link) is that they reduce the individual mental overhead and that a carbon copy of the visual is imprinted in the learner.
There is no punchline to this comment. And I'm certainly not an expert on the matter. I still find the 'movement' fascinating.
[1] https://jeremykun.com/2013/02/08/why-there-is-no-hitchhikers...
[2] http://math.stackexchange.com/questions/743067/visually-dece... (not what I had in mind but this will do)