Hacker News Reader

Comment by claudiusd | Hacker News Reader

Holy crap, great idea! Imagine if adding a "monitor" (virtually speaking, of course) to your setup was no more difficult than splitting a Vim window... that would be pretty rad.

An Oculus Rift is much less expensive than your typical multi-monitor setup too, FWIW.

View on HN

Actually, I'm hoping to be able to experiment with code layout ideas that do not strictly adhere to a window. 2D elements tend to not map well to 3D spaces. One of the problems with the text-in-a-texture method that I've done here is that there is no relief of depth to the text, and relief is actually an important part of how we make sense out of objects in 3D space. It's fine on a 2D screen because the context is so restricted that it all stays consistent. But in 3D, it's kind of disconcerting to see completely flat textures.

Like, why do we still use plain text in flat files for source code (for the most part, there are a few exceptions-that-prove-the-rule)? Why are there so many examples of graphical programming systems that have completely failed? Is it because there is something inherent to text that makes it superior for representing algorithms? Or is it because 1-dimensional text maps to our IO systems the best? I suspect there is a healthy dose of the latter, but we haven't ever had a significant "other" IO system to ever test that theory.

In a lot of ways, Primrose isn't even my goal, more like a benchmark from which to start developing other ideas in text representation in VR UIs. I don't know what they will be, but at least I now have a starting point.

> Like, why do we still use plain text in flat files for source code

Information density. Images and graphs are exceptions to the general rule that text can contain a lot more information on one page than graphics can.

Well... if you look at things like (colour)forth (and highlighting in IDEs/editors) -- most people don't actually code in "plain" text (there's colour/typographic hints that indicates meaning). Other than that -- we do tend to still use text to represent mathematics, so I don't think it's that strange that we usually represent code as text. For another exception to the rule: see graphical editors for UI layout, as seen in many IDEs (ie: using graphical programming for representing graphics) -- or 3d modellers like Blender (when you add bones etc, you could argue that you're "programming"). Then there's of course spread sheets -- but they tend to be pretty text-focused -- even if they use layout to great effect.

> Like, why do we still use plain text in flat files for source code

Text transports with full fidelity between print and screen, and between pretty much any system.

Anything beyond text becomes a) harder to communicate (e.g. consider how many people would struggle with even reading out loud maths formulas with symbols they are unfamiliar with; now extend that to trying to talk about representations that have extra spatial information), b) requires extra tooling for far more varied environments than you ever thought people actually code in (e.g. I have an idea; I want to write down code in my phones note-taking app)

Serializing it for IO is trivial. Representing it in a way that allows us to handle the two problems above in a way that makes it seem superior to text in enough cases is a very, very hard problem.

It's definitively intersting, though.

I believe the reason is that graphical (2d) notations are harder to communicate and therefore not standardized. If they are standardized, they are usually very application specific and not really general enough. For example lambda calculus has a nice graphical representation as string diagrams, since it corresponds to calculations in a cartesian closed category. Typed lambda calculus has a similar representation, but with multicolored strings.

Pretty much any problem easily modeled as data flow has a nice a nice representation in an appropriate (traced, braided) monoidal category, examples would be electrical circuits, chemical reaction networks (simple atmosphere models, so called "Box models" are an example), hardware control software (LabVIEW), general ODE/Statemachine simulation (SIMULINK), "business processes" (forgot the name of the vendor), not to mention the various video/audio/shader programming environments.

Probably the intersection between the people capable of building a "generic" programming language for graphical programming and people who know enough category theory to inform the design is zero, even though software that solves subproblems of this kind is very successful. Ultimately you really want at least a 3d graphical language, because you want to express relations between 2d processes (for example the equation a+b = b+a, should be really understood as such a 3d process). Coincidentally relations between functions is something even programming languages like Haskell fail to address, it has type classes, but no way of specifying the equations that are required to hold.

The only language I'm aware of that does that is Magma, which unsurprisingly is based on Operads (essentially multicolored monoidal category with a fixed set of monoidal functors and natural transformations) and was designed by mathematicians and to a lesser extend Mathematica and Prolog.

My wife chimes in here and says, "don't even talk to me about LabVIEW". She's apparently not happy with it right now, haha.

I don't see what you mean about Magma. This is what I've found for Magma: http://www.math.harvard.edu/computing/magma/. It looks like Pascal with Python list comprehensions and sets as a first-class type.

Source code in most languages is a one-dimensional construct. Every source file in the project could be concatenated, the newline characters removed, and it would still compile to the same program (assuming the right ordering, barring white-space significant languages, etc.).

Is that because we are linear? Is that because our computers are linear? I don't think we're ever going to get away from written language as the abstraction of source code. Most (all?) people think in words. But how that language is presented might change. Literal lines and connectors between code elements has sucked in 2D because it just gets too dense for 2D space. Is that because linear things "want" to stay linear? Or is it because we have had to deal with 2D interfaces and it's easier to work in N-1 dimensions when using an N-dimensional interface?

And while I think I've seen attempts at this in 3D, then we have the other issue of 3D interfaces being terrible on 2D IO devices. Never have the two been put together.

But I think that's going to change very soon. A lot more people are experimenting with this sort of stuff these days. Good-enough stereo displays are accessible to everyone (slap your phone in the cardboard box you ordered a set of $5 lenses in). Language design is more accessible to everyone. Editor development is, too (I couldn't imagine doing this project 5 years ago, and even 3 years ago it would have been a very poor experience). There is just a culmination of different technologies that make it possible for just about anyone to hack on all of this stuff.

Somehow, there seems to be a lot of discouragement lately to do so, but that's a different story entirely. I'll just leave it at saying, I hate the phrase, "don't reinvent the wheel".

Well LabVIEW might be not the best example :), but in part that is because it has many quirks that would hopefully be not present in a more general system. Also because most LabVIEW sheets were probably put together by someone that just wanted to get something to work at all and not really cared about the next person that had to deal with the mess...

I still think that a graphical language for this is worth exploring, because it already happens to be the way domain experts think, which is presumably why most of the successful commercial software is targeted at them. See for example: http://math.ucr.edu/home/baez/networks/ for an overview of how all those things tie together.

Regarding Magma here is a PDF, which contains a summary of the theoretical foundations:

http://sage.math.washington.edu/people/cannon/magma-the-lang...

Essentially it is a language that can do type checked symbolic reasoning, whereas for example Mathematica is for the most part untyped. Page 11 explains the steps necessary to extend the system (you have to write a C-library, so that is less then ideal). The reason I bring this up is because relations between symbolic expressions (for example equations) are really something that lives in a higher dimensional space, see for example the Associahedron (http://en.wikipedia.org/wiki/Associahedron). In a 2-dimensional language ordinary programs would be graphs and metaprogramming would be surfaces between graphs. As long as you have only a 2 dimensional surface available, that kind representation probably would not work, but I believe as a conceptual idea it is rather helpful.

I agree with you, that the moment convenient augmented reality hits, we will hopefully see different variations of program representations emerge. Especially complicated concurrent systems would probably benefit from it, since they don't have a sequential control flow.

Finally, a virtual reality so immersive it can simulate a room full of teletypes.

"A mouse is a device used to point at the xterm you want to type in."

A VR headset is a device used to look at the xterm you want to type in.

Hehe. Well, text will be text... on a side note, those interested in this, might also like: https://github.com/brianpeiris/RiftSketch

Reply on news.ycombinator.com