Media for Thinking the Unthinkable
worrydream.com
worrydream.com
"Evolution, so far, may possibly have blocked us from being able to think in some directions; there could be unthinkable thoughts."
Which prompts today's startup idea:
1. Use Silk Road imitators to send hallucinogens to Amazon Mechanical Turk workers
2. Ask workers to solve impossible problems
3. Pass responses into machine learning algorithm
4. Gain godlike insight
5. Rescue humanity from self-destruction
Linguistic descriptions have a key advantage over pictorial in that they represent or reference rather than show. This enables them to be compact, and omit unnecessary detail. (Of course, showing rather than telling is a strength of visual representation).
Yes, you can have a hierarchy of visual systems, and zoom-in or hide. But a fundamental problem here I think is in choosing the hierarchy - that is, the way the system is modularized.
Different modularizations of the same system are often appropriate for different uses of that system, or for considering different aspects of it. For even a slightly complex system, there are a huge number of different modularizations possible, and not all of them are useful. Often, you'll start with a poor one, and eventually have insights moving you towards the ideal one. (Of course, sometimes the "right" modularization is obvious, especially for well-known families of problems).
All this is very difficult. My point is that it is easier to switch modularities linguistically than pictorially, by changing your concepts. Without the right modularity, it's difficult to pictorially show just the aspects of interest instead of the whole picture. In contrast, one can linguistically omit detail by referencing it (implicitly, as a separate module).
Maybe it's possible to do this visually, though I suspect it thereby would have become linguistic!
[Though the above is a counterpoint, I'm very impressed with the talk. He's working both ends of abstraction, with concrete working software demonstrating cool useful practical techniques that, while not universal, would be helpful in many domains; and also framing it within, and using it to illustrate, the deep universal and philosophical idea of unthinkable thoughts. BTW e.g. uncomputable numbers.]
I thought it was interesting that when he compressed the paragraph describing an equation to is algebraic representation (x2 + 2x = 10), we immediately grokked the algebraic version better, because we have loads of experience reading and manipulating the algebraic expression. If I think back to my own learning career, however, I learned the basic tenets of algebra using -- you guessed it -- interactive and visual representation through Hands-On Equations [1]
(now that I think about it, Hands-On Equations serves as a very nice "tldr" for this speech and the related concepts)
On the other side, the pinnacle of hard sciences, geometry, logic, topology, are very visual and symbolic by essence. A proof is only an explanation of a truth that is already there, that some can "see in their mind". I wouldn't imagine being able to follow a math class without a black board, that would be like a piano lesson without a piano.
The words or the language, as they are written or spoken, do not seem to play any role in my mechanism of thought. The psychical entities which seem to serve as elements in thought are certain signs and more or less clear images which can be 'voluntarily' reproduced and combined. .... This combinatory play seems to be the essential feature in productive thought before there is any connection with logical construction in words or other kinds of signs which can be communicated to others
Good thought doesn't have to be linguistic.
I speculate that the reason most of us only think in words because it's the way we're taught. We can hear the word "dog" because at a young age we were told about the association from each letter => sound. "What does A sound like? aaaaah". But what if instead our parents and teachers taught us that A was a picture? A could be a red square, and B a purple box. "DOG" might be a pink triangle next to a red and a green circle. Great, then you start to remember those three shapes as one image, just as you remember the sounds "duh" "aww" "guh" as one. And you would learn to assemble them together in all the same ways. I bet you would get very good at it.
I think that every verbal system could be represented in a visual system by just replacing each sound with a shape. I actually know of people who do this, who do this fully, just not with boxes and circles. The deaf. They can learn how to read despite having no hearing. They will report that instead of using sounds they think in a combination of hand signs and images. This is the only group of people I know that replaces all sounds with images, but there are a lot of examples of in-betweens too.
Vladimir Nabokov, the author of Lolita:
The long "a" of the English alphabet has for me the tint of weathered wood, but a French "a" evokes polished ebony. This black group also includes hard "g" (vulcanized rubber) and "r" (a sooty rag being ripped). Oatmeal "n", noodle-limp "l", and the ivory-backed hand mirror of an "o" take care of the whites. I am puzzled by my French "on" which I see as the brimming tension-surface of alcohol in a small glass.
Richard Feynman:
When I see equations, I see the letters in colors – I don't know why. As I'm talking, I see vague pictures of Bessel functions from Jahnke and Emde's book, with light-tan j's, slightly violet-bluish n's, and dark brown x's flying around. And I wonder what the hell it must look like to the students.
tl;dr you can do everything visually. Or you can mash up visuals and words. The problem of "choosing between hierarchies" isn't actually a problem, because just as you can make a sound to decide, you can make a visual symbol. If this were false then you would have to explain how sign language works.
If so many people are truly free to pursue higher callings, why don't they? Look at reality, not theory. Find some answers to that question and you'll start to discover the wide and complex range of restrictions on true freedom.
It reminds me that the way to be amazing is to do an amazing amount of work.
On one hand, I agree with almost everything he says in the talk, amazed by the prototype tools he presents, the concepts he discovered to make those tools usable. And I can see that in each talk he has some new insights to present, deeper connections, better and more general creations - and thats really exciting.
On the other hand, his attitude that he doesn't plan to turn the tools into products, that they're just prototypes, that he expects someone else to come and create actual products - that rubs me the wrong way.
The thing is, we've found excellent collaboration platforms exactly for these kind of prototypish things that we don't plan to make a product of yet we would like other people to toy around with the general idea. The most recent incarnation is GitHub.
So why not publish some of these prototypes? Its not about the code, its about forming a community that will build upon those ideas. Its about kick-starting the construction, setting it in motion. As far as I can see, that is what Victor is hoping for - people taking these ideas close to their hearts and building them in reality.
Is the idea not ready yet? Not exactly polished enough or general enough to be published? The visualization drawing / exploration tool looks like an awesome start to me...
I don't understand.
Edit: Oh wait. https://github.com/damelang/nile contains the presented Nile viewer. Nevermind.
"An 'Industry vs Research' dichotomy that I sketched at a conference. I probably should elaborate on it sometime. https://twitter.com/worrydream/status/392335781374619648/pho...
One can understand why, given the options as viewed by Victor, one would choose "research" if given the choice.
Left side of the board:
- No need to ship anything. Its possible to just put whatever you have out there and let people play with it by forking it. It doesn't even have to "build", e.g. other people could start by making it build reliably. There really are no limits.
- All code is immediately applicable for builders, even prototype code. Immediately applicable means that they'll be able to read it, study it, toy with it (possibly interactively), see how it works, make some changes => get inspired
- github is not results-driven. No pressure to "ship it", on the contrary - if you want to ship something, the common path is to fork and get it there by yourself, right?
- build and exit - the exit part doesn't even apply.
Right side of the board:
- Source code does export knowledge to be built upon - and its even more knowledge than just old-fashioned media (videos and articles) - it exports hands-on, interactive, reshapeable, remixable, engaging knowledge.
- its perhaps not long-term enough, but on the plus side it actively engages a much larger number of people.
- it could indeed distract from deeper, foundational work but I think there is enough room for anyone to choose what kind of work they want to do - no need to centralize it all. For example, people could "pull" new foundational work from Victor into their own projects.
- its definitely exploratory - massively exploratory as it engages more people actively on many levels and causes an innovation ripple effect.
A tiny example: as soon as V8 implemented generators in JavaScript, the node.js community started experimenting with async runners. Lots of async runners were produced and ideas are still studied, analyzed and exchanged between various projects. But before V8 published it, it was all just excitement and discussion about the possibilities with zero hands-on experimentation and a much smaller number of ideas.
- it is indeed ongoing - nothing needs to be shipped, no project need necessarily be done.
Additionally:
- it represents extreme transparency - everything open to the public - which nicely matches the scientific method.
Percieved but non-existant downside:
- personal recognition is harder - not quite true. Its easy to think that some things may get lost in the mass of people, but usually the contrary happens - the pioneers always get the recognition. [citation needed - this one is based on personal observations]
Possible downsides:
- academic recognition: no idea how academy (and related publishers) treat open-source in general. It seems like they would be a perfect match for each other as they share a lot of common principles, but so far it hasn't engaged researchers more massively...
Its a third category - different from traditional research and different from industry. I believe its worth considering.
What I yearn for is about 20 more Bret Victors to appear in such a way that they'd all get together in the same room for a few weeks and do some cool shit. Maybe thats where he's headed?
Inventing on Principle: http://vimeo.com/36579366
Stop Drawing Dead Fish: http://vimeo.com/64895205
Drawing Dynamic Visualizations: http://vimeo.com/66085662
The Future of Programming: http://vimeo.com/71278954
Meaning: my best ideas always come from playing in non-traditional representational systems within a certain medium. Particularly drawing. I find sketching/scribbling/drawing things generates ideas that I could not have thought of by just writing/brainstorming. You may even be able to apply it to the example of how software development models have jumped paradigms into mainstream business management (eg, agile teams). Bret's onto something here.
The electrical drawing would be obvious to anyone who is used to working with them.
There are obvious places where this would work– dissemination of material is always useful in many different forms. But this is NOT an answer to a question that is being asked. Rather, it is an alternative form of dissemination to an alternative audience.
Back in high school I played a musical instrument. I practiced my scales, was quizzed on them, then practiced them some more for years. I got pretty high-school-good at this instrument, enough to earn a small scholarship. I never continued it into engineering school though, and one of the milestones I never reached was the ability to improvise within a key. You see, despite practicing the fingerings of the different keys hundreds of times, it wasn't enough to gain the intuition needed to just feel it out on my own.
Those people who read that paper and could decipher what the authors had written HAD practiced their scales enough times to develop that intuition.
Bret isn't trying to provide the most compact representation out there. He's trying to present the best model that helps you build that intuition the fastest. In music, you had to spend hours repeating the same scales so your fingers would get hard-wired into the valid combinations for a specific keys... the question is how can we build tools and techniques that help us reach that level of effortless mastery faster.
"here, they are discussing some relationships: the regular latice, L grows linearly with n, we see L growing linearly with n, and we see C is staying constant. In the random network, L grows logarithmically, and we see L growing logarithmically, we see C going as a reciprocal relationship. When we read the word "logarithmic", we don't need to reconstruct that relationship in our head, you can just see it"
Does the average reader of Nature need to be shown a graph of what a linear, constant, logarithmic, and inverse relationship looks like? I don't think so.
That said, I imagine there are highly terse and technical ideas that cannot be simply represented in the current mediums of publication and which could benefit greatly using a presentation using an interactive widgets, making more complex ideas more easily communicable.
Giving a slider where N can be visualized over time shows how changes in N accelerates y=x^N away y=Nx. These compound complexities better seen in interactive widget as the average mind is not well equipped to visualize them.
Sometimes I feel like nerds kind of enjoy the exclusivity provided by "cryptic" notation. What Bret is doing is not genius level stuff, he's just doing what a lot of people don't have the balls to do. Attempt to make "hard" things easy enough for "regular" people to understand by appealing to the right side of their brain.
But I think Bret's point isn't that every research paper should be as verbose as an introductory textbook, but rather that interactive widgets can express ideas which text and static images in published papers can't.
This. I had such hopes for iBooks, but then making widgets is hard, and making a Microsoft Word for widgets, harder still.
This is not just about simplifying scientific notation so it is easier to digest for an untrained audience: it is about using tools to remove as much cognitive "shit work" as possible, so that our minds (including those of experts) are free to think the "unthinkable thoughts" of the title.
Learning is hard, but that is a good thing. It's like exercise. The more you work at it, the easier it becomes.
But those who do not have the necessary math background - this will not really help them at all. Sure they might have some understanding of this particular algorithm - but when it comes to abstraction and extending that knowledge to other problems I do not believe it will scale. You do not have the fundamentals to think creatively (in a mathematical sense).
The circuit for example. They might now understand how that circuit behaves. But if I show them a completely different circuit can they infer how it behaves? Likely no, they were not able to pick up on all the non-linear behavior that occurs within circuits.
The author gives a good example here, if you choose to read it:
http://worrydream.com/MediaForThinkingTheUnthinkable/note.ht...
Basically, Claude Shannon was able to create his information theory because he could use things like calculus, probability, stochastic processes and graph theory to aid his thought process. He needed those tools to be able to think things that would have been unthinkable without them.
Years ago, before there were power tools, all furniture was crafted by hand. Painstakingly. It would take a craftsman years to develop his craft. Eventually though, power tools lifted a lot of the learning curve. It became easier and faster to create furniture. Eventually a lot of those power tools were replaced with automated CNC machines. With these machines you can pretty much throw anyone from the street who can read on the machine. Productivity has increased so much that you can pretty much buy furniture these days for not that much more than the cost of materials alone! It really is an incredible statement.
Now not to take the metaphor to far. I realize you can't turn science into an industrial process... it is a creative process as much as anything. The main point is that as you lower the learning curve, you enable a few more people to gain the new skills required.
Again, i'm not suggesting this was ACTUALLY the point which motivated him to come up with the concept. But to me, this is the really cool potential outcome.
Not only does it reduce the time to create, it reduces the skill required, the flexibility of product, and the overall level of craft.
Ignorance is not something we should aspire to or assist others in attaining.
Representing simple, familiar relationships visually and interactively may indeed not offer much clarity, just as a single brick may not offer much shelter.
His point is, this average can change, and its the quality of the symbols which deduces the set of readers.
I am somewhat intrigued by the idea to represent arbitrary systems. I wonder where you might hit a wall within the editor, and how you can extend the editor to be able to visualize what might be possible to vizualize now. Like Emacs paired with these capabilities?
ALBATROSS gave me the chills.
This reminds me of another tool, Korsakow[0], which attempts to do a similar thing in a different way with documentary material, by presenting it in "shortest narrative units" or SNUs that relate to each other. It is one of a number of "interactive documentary" tools (like Mozilla's Popcorn [1] to some extent) but I wonder if it could be repurposed to "explode" a talk in a slightly more digestible way that could be reproduced by those of us who don't have Bret Victor's skillz.
[0] www.korsakow.org
[1] www.popcornjs.org
I think these tools could be especially useful for helping people understand systems like climate change.
It's on a 2D graphing library but his visualizations are incredible.
> Thanks! The tools I showed were just prototypes that I made for the talk. I don't have plans to make them into products myself. (But someone else might!)
In the meantime, if you're looking for a free, easy-to-use circuit simulation tool, I've used LTSpice (http://www.linear.com/designtools/software/) in the past for my EE classes. The Windows version runs pretty well under Wine and the UI isn't too shoddy. It's pretty far away from Bret Victor's vision though, as you need to rerun the simulation again in order to see a new plot of the node voltages and currents.
Further, on an individual level, each person might be able to think what maybe they personally couldn't think otherwise. And in that, communication might happen which gives rise to new thoughts.
Victor didn't complicate the process. He used that quote to illustrate his claim that it's precisely because of innovations in symbolic representation (i.e., representing the equation as x^2 + 10x + 39) that you are able to think of the process as simple.
[1] http://en.wikipedia.org/wiki/Mu%E1%B8%A5ammad_ibn_M%C5%ABs%C...