Dynamicland: The Next Big Thing Is a Room
phenomenalworld.org
phenomenalworld.org
- The community! Most people I interacted with were pretty inter-disciplinary. Some came from an education heavy background, others from a more programming language background. Some from interface design, others from physical installation or video game background (and everything in between). Everyone had their own degree of skepticism but were excited by the discussions that were happening (a sign of a good research culture!).
- The representations: What excites me the most about Dynamicland is that it's an environment for fostering unique representations of ideas. The cultural forces & ideas (social programming, remixing, visible state / code at all times, 3d environment, etc) baked into the space encourage the experimentation & creation of new ways of representing complex ideas. Some scattered examples here - https://twitter.com/dynamicland1?lang=en
- Removal of artificial barriers: When programming on a computer these days, there's so many barriers to doing simple things. The amount of code that exists to do virtual actions in a virtual world is MASSIVE, and acts as a huge barrier. This is something Bret talks about at the end of his interview on Postlight (https://postlight.com/trackchanges/podcast/computing-is-ever...). Because code / programs in Dynamicland are embodied in the room, you don't need code to move a dialog box or a slider around. You just moved it yourself. This is super powerful and it means your code can focus on the actual computation itself, not on virtual UI movement. Eventually, Dynamicland will have robotics to automate moving of objects, but this already is a great start.
- Moving around: Moving around, even if it's just around a table, is AWESOME. We're so used to sitting (or standing) at a desk and staying still that we don't get to take advantage of embodied intelligence. We think in SO many different ways, and our "thinking" doesn't just happen in the head. Our arms, legs, stomach, and feet all contribute to the thinking process. Combine that with multi-sensory representations that live across all of these channels and you can explore a thought & idea space SO quickly and uniquely. This is so hard to describe and under-rated. This link (http://worrydream.com/ABriefRantOnTheFutureOfInteractionDesi...) and this talk (https://www.youtube.com/watch?v=agOdP2Bmieg) attempt to do these ideas justice but it's quite hard to transfer this context!
It seems like they're mostly seeking funding from larger institutions, as crowd-funding such a large research effort isn't really feasible or sustainable.
I jumped around a skimmed a bit, so maybe I missed it, but are they prrogramming for the room itself only, or in a framework/library that allows for this, or just in a similar language? Is it all just Javascript and a well defined hierarchy of objects that can be manipulated?
How do two people, working on two separate programs, share implemented functions "for free"? The sliding across the desk portion is irrelevant (well, it's cool, but for this specific question it's no different than sharing a gist link I think), what I'm wondering is the details.
So, yes it's a shared language, and yes, it's a shared framework (the language embeds the framework concepts as first class entities in how it functions. That makes sense, and it also explains how some output display function (which would be fairly generic in this context) would be easily shareable, and by the nature of the platform, also immediately shareable if done in a certain way.
It is similar in concept to how Javascript ona webpage has access to the dom, and the bindings for the dom can be expected, so you can write something that transforms a <table> in some way, and expect it to function similarly is other tables are provided. But it might be even more accurate to say it's like CSS, where CSS and the DOM are so closely linked that (at least from the perspective of CSS, if not HTML elements in specific) there is no interop layer, CSS is meant to apply to an HTML document, so it's designed with that in mind and the interop layers are for the most part nonexistent.
So, in that respect Realtalk is sort of like CSS (with more programmability, I think) for this environment of sheets of paper that support input and output. Very cool.
Again, I only used it once (and I made a DynamicLand compiler in DynamicLand! Video here: https://twitter.com/i/web/status/963497112284512256 ), but if I recall correctly, the main way to "share functions", was to actually just load someone's code and copy it. Partially because the language/editing seems very new still, and perhaps partially to encourage this sort of "real world sharing" vs. virtual sharing. I recall just "forking" files a lot and making new copies of things (kind of like "Old" folders I used to make before version control).
Because truly plug-and-play components are an awesome idea. However they are not a new idea at all. As far as I can tell the real reason they are not more popular is because us programmers are worried we will be accused of being users if we use tools that are interactive. See Visual Basic 6 (which was an amazing system). Of course programmers do not realize this psychological issue exists and will not admit it is a possibility.
But in the context of Bret Victor's lab with the projectors and little pieces of paper, that puts plug and play components in a different category that makes them more palatable.
Also, projector based AR is much more convenient for people to demo than HMDs, but also not very practical for widespread deployment. But AR is amazing in general and this allows them a friction free way to demo its power with components.
But I think that multiplayer real-time collaborative interactive component based AR programming should definitely be a more common thing. It combines the advantages of all of those techniques. I also think though that all of those things are useful in lesser combinations. I believe a big part of the reason at least some of those things are not used more or more effectively is cultural or psychological rather than technical or practical.
The thing is the more you take advantage of components and interactivity the less you use the complex colored encoded text. That means programmers can spend a significant amount of time snapping together components and configuring them. Unfortunately programmers are not able to recognize this as programming. We have a feeling the more we do it the more we may lose our special incanter status and be classified as users.
It would be nice if you explained what you even mean by this instead of just claiming that programmers who don't agree are in denial.
What programmer is embarrassed to also be a user? And programmers use interactive tools all time. REPLs, debuggers, editors, GUI design tools, and live-reloading web servers all qualify in my book.
To give an example: if you want to replace hierarchical filesystems you shouldn't build a thing that hides them from me, you should build a thing that doesn't use them at all. (This isn't a hard rule, sometimes exceptions are worth the trade-off.)
Understandably, this means changing things is a LOT more work than you're probably prepared for. Don't assume that we're only stuck in the past for bad reasons.
this is a specific example regarding labview, but these types of biases exist everywhere, where engineers blind themselves to what is possible by the opacity of what was possible.
Maybe someone has to build an Open competitor so that we can address problems mentioned by other commenters. Maybe I'll do that here in my hometown and share it with everyone.
Dynamicland has 2 components - the paper and projector idea which is bad, and the interactive OS and systems which are fantastic. Replace the paper and projector with AR glasses and you have a winner.
AR allows exactly what you want - real time dynamic collab 3d interactions. Paper does not. It's 2D, flat, limited, with occlusion. You've just taken 2D screens and reversed the light source.
Dynamicland is a sunk cost. Drop it fast. Move to AR.
I'm talking above about full body standalone/wireless headset AR (like magic leap) not "point your phone camera at a QR card" AR. Full body is active and immersive. You can jump, run, turn around, and it tracks and models the 3D space around you in real time. It really is something special, and Bretts talents in dynamic collaboration are exactly what is needed in this new tech field.
Instead he's huddled under projectors pinning paper QR codes to chipboard like it's 1980 when he could and should be pushing the frontier of interaction and design where his work can make a huge difference.
Essentially, he asserts that computing (or computer science even) isn't a real field (like biology or physics). Most people can't name the early computing pioneers whose work they build on (not true in Physics or Biology, we celebrate the pioneers) nor are they familiar with the work that was done in the Xerox PARC days.
It's gotten so bad that a lot of computer science research just assumes that what we have now is what's going to remain forever. In the 70's, all kinds of interesting ideas and experiments were tried.
Hardware implemented VM's, direct manipulation programming, OS-free computing environments, highly re-configurable computing a la FPGA's, and hundreds more.
Nowadays, we think that creating custom ASIC to run machine learning algorithms quicker is innovative and novel.
All this to say that many of the ideas in Dynamicland aren't new. They're rooted in ideas that are decades old. If you look at Bret's papers he uses as references frequently, you'll notice how many of them are over 10 years old - http://worrydream.com/refs/
I really hope this isn't how we teach programming to the next generation, because it will severely limit their understanding of what makes programming great.
Possible application: interactive restaurant menus. Some places have tried handing out tablets, but you have to have someone hand them out and retrieve them. The system has a camera, so have it recognize open space on the table and people in seats. Present people with menus projected on unused table space. Plus you can offer customers games while they wait.
(Inevitably, someone will put in ads.)
Dynamicland looks like a real world IDE, where a lot of the interface consists of physical objects.
It's more like a Mathematica notebook than a bash shell.
Ya lost me. I hate cooking specifically because of the physicality, of the different tools, etc. If I liked cooking and all that shit, I'd be a chef.
It’d a pretty dull existence if the only things you liked to do could be determined solely by your current job
But how will it work when non-creative people take the technology and make it bland — and infuse it with ads, because that is the only viable business model that tech seems to be able to come up with.
They want to mirror the Carnegie library model as much as possible. They're also non-profit and not interested in building a product that can be purchased (instead they think of this technology as infrastructure available to almost everyone, like the internet or running water).
And yet it gets to the 1st page on HN, no one complains, and my comment will get huuuge number of negative points, I know, as usually when someone posts a comment about this kind of bad design of a page/author people worship.
The fact that it has a lot of upvotes shows that aparently many people don't have issues with the presentation.
Anyway, my browser has a "reader" mode that lets me make any website readable with a single click, so I've stopped complaining about hard-to-read aticles. Better to focus on the substance of the article...
I recognize the limited Value of Dynamicland as an exercise environment, maybe like Scratch. Even in that form, special considerations would need to be put so that disabled people have an alternative way of doing the same exercises.
Dynamicland makes a lot of other things, not related to accessibility, harder too. For example, how do you deal with larger codebases or distributed teams? How do you quickly collaborate on one piece of paper while being in two different places? Very simple to do for a Google doc. Almost impossible to do here.
The idea is kind of interesting but it would be only worth it in a pre-internet age. In 2018, people expect realtime collaboration with people in different parts of the world and with Dynamicland, that's not going to happen.
I suppose it's inevitable that if this vision of the future takes off, we'll be second-class participants. But I guess it's too much to ask the majority to not work in a way that would be great for them, just because it's a new challenge for a few of us. We'll figure it out.
There is no reason why Dynamiclands can't similarly have specialized hardware and software for people with various disabilities.
Plus, a more natural/human interface might be more _accessible_.
In other forms, paper should be a representation a user might choose, not the main form of storage and manipulation. Same for other physical media like the colored poker chips and pebbles mentioned in the original post.
Most representations are haphazardly created (math notation, periodic table of elements, etc) by (often) singular experimentation. Representations are poorly researched and understood (there's no branch of science dedicated to them). If representations were studied more closely, we could get much better at deliberately creating representations that are more humane. When the scientific method is applied, we can then apply the engineering method to create new things from those principles. Bret's talk is a great, accessible overview of these ideas from his context (https://www.youtube.com/watch?v=agOdP2Bmieg).
To vastly oversimplify things, one way to improve accessibility is to parallelize representations that live in just 1 or 2 sensory modalities to all of our sensory modalities. This way, math notation could be made accessible to & equally powerful to the blind (as it is to those who can see).
I empathize (a little, it would be naive for me to assume I can 100% empathize) with your reaction along the axis of accessibility. What Dynamicland looks like right now isn't what it's going to look like later. Right now, you may see Dynamicland as text boxes but now distributed in a room. In 50 or 100 years, there may be no text AT ALL and you can use the senses available to you to interact with information.
To paint another perspective, I would argue that current computing environments are VERY in-accessible for many people. Laptops and desktops only tap into our visual and symbolic channels, with some tactile feedback (but quite minimal, we know where "X" is on the keyboard b/c of spatial reasoning not b/c the X key gives us specific/unique tactile feedback). Humans who thrive in spatial and kinesthetic channels are debilitated on a modern computer.
Interesting; this isn't something I know anything about. Is it not feasible to develop alternative hardware and software that adapt our symbolic notations for these people, while still enabling those of us who practically require symbolic notations to collaborate?
In general, we're just not great at processing symbolic representations. We haven't evolved to do so (unlike spatial, visual, auditory, etc senses for matching representations) and have only manipulated symbols for a few hundred years. I mean true abstract symbols, not hieroglyphics or other old symbols that still mapped to language / were still quite visual.
I believe it could be more compelling to figure out how to re-represent things instead of just scaling up symbolic representations. That helps make it more powerful and accessible, for everyone.
A good example here that Bret mentions in his talk is Roman numerals -> Arabic numerals. Or even typing paragraphs of text to Arabic notation for equations (y = x^2 + 1 used to be PAGES of written text). People used to think only super educated elite mathematicians could grasp algebraic ideas. But after they were re-represented, we discovered almost all 8 year olds could grasp the ideas!