Wio: A Clone of Plan 9’s Rio for Wayland

The Plan 9 emphasis on per-process namespaces corresponds to the idea that a new graphical application is given a handle for a framebuffer that corresponds to the size of the window it can draw in. The Rio idiom has every window start by default as a textual terminal, so it's natural to be able to take over that framebuffer to run a graphical app.

It's a bit like the situation on older systems, like Sun workstations, where the textual console and the graphical display were only informally separated: sometimes the X server's display would be interrupted by lines of text printed onto the screen by the kernel's console driver.

Or even a web browser... First you open a tab, then navigate to the content you want to display there.

Why? For example, people doing video or audio production sometimes create extensive personalized HID setups, with knobs and sliders and joysticks and touch screens and drawing tablets and etc. Even with expensive and clunky tech. What if everyone could easily have better at need?

> gorilla arm [..] mouse [...] the only thing I'd want to use for real work is

I too like a standard of "use for real work" without compromising existing tooling. I prefer a well-tuned little red ThinkPad TrackPoint to mouse, in part because it requires less movement. But even with all of Point and touchpad and mouse...

What if your entire keyboard and adjacent frame could serve as a multitouch surface, without compromising keyboarding experience? (Well, compromising it beyond nice ThinkPad chicklets.) My very limited experience suggests it could be nifty. I was playing with a completely-non-functional "sanity check" prototype, got distracted surfing, and then realized I'd become puzzled and distracted... because stroking the J keycap was bizarrely failing to scroll the page. :) Suggesting that adoption might have a low barrier. I think there's a startup hoping to do a plastic overlay, like a keyboard protector, that does multitouch.

> monitors

And those monitors could be 3D monitors. Perhaps adding subtle Z offsets to font color and weight and spacing and alignment. And semi-transparency becomes more powerful. All 1M NPM packages fit on a 1080p screen as color dots, but with high-res 3D, they might be little lines and shapes, with over and underlays and connections. 3D datavis infusion.

Having full hand pose could permit much richer hand-barely-moving gesture vocabularies... than press-hold-and-release Ctrl-Meta-Mumble.

But for expert use... When smartphones were new, the user population untrained, and onboarding was critical, there was the idea of skeuomorphic user interface design. The app should resemble the real-world object. So affordances would transfer from novice user experience. The calendar app should have lined pages, a spiral binding down the center to hint at horizontal flipping, and textured leather margins. :) We don't do that anymore.

For expert users, what are the odds that the severe constraints of physical reality line up nicely with the domain constraints desired for maximum productivity? How long before people start complaining "I don't like reality - no stretchy arms - you actually have to walk over to something to grab it - yuck".

I joke that I currently don't do V of VR, because I almost always have camera passthrough, don't do A of AR, because I'm usually ignoring the world other than for user input, and don't do R of either, because the UI is so often aphysical. Resemblance to physical reality as design smell.

Though how to "think with your hands", when HIDs are still so crude? Tracked/instrumented manipulatives could be neat.

Ah, I have tried quills a couple of times. And dip pens. It was interesting. ;)

> movies [...] can't help laughing about the ergonomics of it

Yes. Though they do seem to have an "outreach" role, raising awareness and interest that improvement over the familiar is possible.

And moving while working, like shifting from sitting to standing desk and back, and walking, can be nice.

> minimize cognitive load, eye strain and arm strain

Nod.

> focusing on a screen a couple of centimeters [...] eye strain

The focus distance is usually either around 1 m, or infinity. Downsides include being fixed, creating vergence-accommodation conflict. Prototypes, and the new Microsoft Hololens 2 (which paints the retina), variously improve on this.

> minimize cognitive load

One thing I've played with, is taking a downward keyboard-cam of hands on keyboard, and overlaying it on a 3D monitor. Raising it slightly in Z seemed to allow focusing past it, making it merely bothersome, rather than untenable. Upside is it reminded me of an old Lisp Machine, with the screen telling you what various keystrokes would do in your current context, so I don't have to remember them all.

> less hand movement

Yes. I really like the idea of keyboard as multitouch surface. Stroke a keycap, rather than moving to a touchpad. Touch a modifier key keycap, rather than having to actually press and hold it. And optical finger tracking allowing subtle hand gestures to be commands.

> read and write documents [...] 3D space or are projected in thin air in front of me is uninteresting to that end

Hmm. Sometimes it's useful to print out a document, and put it up on a wall, so you can see the big picture, or talk with someone about it. If one has a task which fills the available monitor space, it's easy and common to remember what it was like with a smaller monitor, and not want to go back, but harder to imagine whether the same might happen again with further expansion.

More generally, one thing that cheaper real estate might do is lower the barrier for tooling to provide speculative assistance. You're writing for an audience including someone from Great Britain, using a phrase that has different meaning there than in the US... checking that with a keystroke, or as part of a "I'm done, check it now" report, seems less attractive than "comments are available off to your side - glance if you care". Grammarly meets VR.

For myself, writing documents is often preceded by a lot of graph drawing, as I figure out the structure of the domain, concept space, and presentation alternatives. I very much look forward to moving that into direct-manipulation 3D.

I used to resort to firebug light (DOM viewer) bookmarklet but it doesn’t seem to work anymore.