https://youtu.be/JXv8VlpoK_g
I wonder what was the motivation behind this project and why it never really took off. Also what are the people behind it up to nowadays - does anyone here know the backstory to this?
I think there were two reasons for this:
- It looked cool, even if sometimes very impractical. Now it's a bit mundane but at the time it was like breaking the 3rd wall, we were so used to the flatness of the desktop that it was really weird to view it as a 3D object.
- Apple and Microsoft were in no rush to bring these features to their commercial OSs (and proably for good reasons, MS tried to have the 3D carousel in Vista but even that wasn't very useful). That meant that people in the Linux world in particular could show off "hey, can your Windows XP do that?"
But of course eventually we realized that 90% of these features were counter-productive so we only kept the bits that made sense (ability to scale down windows in real time easily to make thumbnails or previews, faster rendering, a bit of transparency etc...).
I remember, circa 2005, there being compiz extensions that enabled something similar.
Virtual desktops which I believe linux supported long before spaces acquired a nice visual metaphor with compiz which allowed you to zoom out to see them all and drag windows between one and the other.
Windows could be set to have a small amount of resistance when passing other windows to make it easy to stick them side be size.
The effects for windows creation and destruction were quite cool and super configurable.
It also reminds me that I used to tweak a bunch of settings in OS X to speed up and disable animations, but I gave up years ago because Apple kept breaking them. Now I just use OS X with its default settings and stay away from things like Spaces that insist on cramming animations into my attention. That makes me sad, so I am going to go back to not thinking about it.
Recently found the "reduce motion" checkbox in System Preferences > Accessibility and instead of sliding it just does fade-in-place which is way less annoying and I think feels faster because you start to see the other window right away, even if there's a window fading away on top of it. The slide from desktop to desktop feels like I need a split second to orient myself after being spun in a chair.
https://www.youtube.com/watch?v=EjQ4Nza34ak
Some good quality images of it running with my blog in the old days:
https://www.javipas.com/wp-content/lg1.jpg
https://www.javipas.com/wp-content/lg2.jpg
Desks provide much more creative value across the first two dimensions than the 3rd: you want the stuff you're working on within sight, touch and arm's reach. "Piling up" is a "storage" or "attention reducing" strategy because you can't work with the piled items or the content with your hands. You largely don't care what they even are unless you're working on them... so pushing virtual representations of work surfaces deeper down an imaginary z-axis makes little sense.
The demos in the video look fun and exciting, and probably justified the applause at the time. I think the desktop metaphor could be worked over. Tablets and phones dropped it from the get-go. But I'm yet to be convinced that visually arranging data across a projected third dimension except when you want to draw the user through (like in a game) adds much to the experience.
The flipping and "piling" you get in Windows and macOS seem like decent uses of 3D. Those effects basically just add+hide 2D surface. It's the vanishing-point stuff that seems gimmicky to me.
While not entirely convinced myself, I was persuaded to think that there might actually be some value in this after watching the various videos where people have built very large functional constructions in Minecraft. Watching the creators explain what they've done, and seeing how easy it is for them to navigate even tremendously complex and interlinked structures through reference to the relative positions of different pieces certainly makes it seem like there's something to it to me.
I was following this back then -- Apple had just come out with 3D manipulations in the desktop (e.g. turning a widget window around in dashboard around to view their configuration on the "other side").
Some engineer at Sun, in their spare time, put together a quick demo with 3D windows (using Java/OpenGL bindings IIRC). Basically a quick play on the idea of windows being objects rendered in a 3D space by a compositor (which OS X had introduced to the mainstream a while before).
OS X still rendered them as 2D projection, except for the occasional effect like the flip-around in 3D space, while this demo had them floating by default (but still only shown the same effects, flip around to write notes etc).
The Looking Glass demo was very basic, it got some talk in forums at the time, and it was even presented a few months later in a SUN "keynote". For a while they pretended like they had something there, instead of just a half-arsed demo.
There was never any big project around it, nor much thought. The highlight was ...turning a window around to write notes, configure, etc (e.g. a copy of what Apple had done commercially and already shipped with minor changes).
Even the very concept of Windows in 3D space is not that novel, Microsoft had done something similar before Apple, as well as others.
It didn't went anywhere, because (at least as implemented this far) it doesn't solve any problem better than the regular desktop.
In short: Looking Glass never went anywhere because it was a proof of concept by 1-2 Sun employees, when window compositing became possible in XWindows/Java.
Remember, Sun didn't do well after the dotcom boom and Sun wasn't cheap but Sun did invest a lot in desktop UNIX (Gnome HIG, documentation, internationalisation, and a major contributor with LOC). Sun GPLed this and many other things (among which Java itself and Solaris) right before they were sold out (to Oracle). We can only thank Jonathan Schwartz & Co for that because Oracle would've kept it proprietary (speculation though).
It isn't very innovative. It was one of the many window managers which did this. 3dwm, 3d desktop [1] [2] (from around 2002 IIRC), Beryl, Compiz, and Enlightenment (E17 & onward) are some other examples though not all of these were 3D accelerated.
As you said, the problem with this is its gimmicky; not productive. In gaming nice 3D effects can add something to the experience (immersion) but on a desktop it shouldn't be very noisy or abundant. So after the initial wow-effect was over these effects didn't get a long lasting stay in products.
Things like that being gimmicky is not the major problem. The major problem is they rely on 3D hardware which (or its drivers probably) is always more or less quirky. I have not seen a single PC on which Compiz would perform reliably without occasional crashes and/or artifacts and I can see no practical reason for a window manager to use hardware 3D as window decorations are not that hard to render, alt+tab task-switching 3D cube is hardly useful and wobbly windows effect is not just useless but also annoying. I believe there should be a category of "wow! cool!" window managers with all kinds of Hollywood effects for fancy demos, cool kids, drivers debugging and concept experimentation but no serous distro should rely on them the way Canonical did cementing a Compiz into Unity.
At the same time I would love to try a WM like the one by the link running in a VR environment and controlled with old-school cyberpunk VR gloves or something.
https://web.archive.org/web/20000822012848/http://www.oreali... https://web.archive.org/web/20000822013028/http://www.oreali... https://web.archive.org/web/20000822013204/http://www.oreali...
Source code: (https://web.archive.org/web/20000131021557/http://www.oreali...)
My friend Nick McKinney and I borrowed $25k from a chiropractor and spent a year trying to get Linux users to replace X with Synapse (which was a paid offering that wasn't compatible with anything.) We had no idea what we were doing, no idea how to get customers, etc. So we gave up and released it as open source.
It was a lot of fun! I have tons of stories, but I gotta go because I am trying to prepare for a talk (currently posted on http://js.la/)
Xgl [1] was released in the same year and had quite similar capabilities [2]. Xgl, in turn, lead to Compiz/Beryl which is the technical ancestor of the most popular window managers we run on Linux desktops nowadays (e.g., Kwin in KDE).
So to some extent, we still use the tech which was developed back then, the effects are just a lot more subtle.
There's acryllic everywhere too, which is blurry transparency, but you might not notice it either.
That said: I miss Compiz wobbly windows. They were cool.
Last I checked, Kwin still had that functionnality.
That said, I just found https://www.stardock.com/products/windowfx/
Because I get the impression this (not-C, not-Rust, not-stereotypically-fast) code is what's running at 60(?)fps, doing the actual deformation animation.
Nice.
Alas, the improvement that should really be made is to encode both n and n+1 in the same update and interpolate in the vertex shader, not hard here but I was lazy..
The difference is that both Looking Glass and Xgl works as an proxy X server. While AIGLX was an effort to make client-side libgl implementations usable from the X server itself, which is good for Acceleration of GLX in Indirect mode (hence AIGLX) but also as a general HW acceleration backend for things like XRender and XVideo (for example intel driver emulates XVideo in terms of OpenGL) and with addition of one simple OpenGL extension (ability to create GL texture from contents of X11 window/pixmap) allows all the 3D window manager tricks.
At the time it was pretty cutting edge and the demos made it seem very polished and useful. In actuality the live version was rough round the edges, fonts and rendering looked soft. It was definitely a proof of concept and didn't really have much breadth beyond the few use cases presented in the video above.
That said yes, it was very cool but even back in 2006 I and my classmates were questioning quite what you'd do with such a product. It seemed like a solution in search of a problem.
Also OS X 10.4 Tiger was when Apple's Mac platform became very solid indeed and started making inroads in academia and enterprise. Most of our professors for example switched from Linux to Mac around then.
I think Looking Glass failed then not just due to legal pressure from Apple but competitive pressure too. I also doubt Sun - which if memory serves had financial issues and was suffering diminishing sales of their SPARC platform - had the clout or will to invest in this particular project.
description from wikipedia
BumpTop was a skeuomorphic desktop environment app that simulates the normal behavior and physical properties of a real-world desk and enhances it with automatic tools to organize its contents. It is aimed at stylus interaction, making it more suitable for tablet computers and handheld PCs. It was created at the University of Toronto as Anand Agarawala's master's thesis. Anand Agarawala also gave a presentation at the TED conference about his idea. The 1.0 version was released on April 8, 2009, along with a fully featured pro version as a paid upgrade.[1] On April 30, 2010 the author announced that BumpTop was being discontinued and that they were taking the software "in an exciting new direction."[2] Two days later, it was announced that the company had been acquired by Google. On January 5, 2011, Google released a sneak preview video of Android 3.0 Honeycomb[3] showing a 3D desktop with features purportedly taken from BumpTop.[4][5]
I see two ideas for how it might be useful that are practical: Putting windows "out of the way" when they're not being used, and attaching notes to files.
For the first, I'm not sure the 3D works well with how I like to work, because I like to use up all my screen real estate, and a bunch of floating tilted windows are still in the way (because they take up a bunch of space - maybe 15-20% of the real estate of the full window, in the examples from the demo) using this approach. On OS X, by contrast, I have several options, all of which really do get the window completely out of the way - I can hide it, I can minimize it, or I can stick it on its own desktop. Throw a window tiling tool like Spectacle in there, and I get a pretty large amount of power, while still keeping things simple.
Adding notes to files by writing on the back of the window feels odd to me, UX-wise. To view the notes, do I have to open the file and then flip the window around? How do I know that a file has a note, without doing that? I think I'd rather just have a "notes" bit of metadata that is viewable in the file manager.
In general, I think that 3D is maybe just awkward for organization. Evidence: Out in the real world, we have invented all sorts of objects for 2d-izing - or even linearizing - things for organizational purposes. Shelves and file cabinets, for example.
I looked through the video and I didn't see anything I want to use. Some concepts made their way into some current desktops (the dock, floating windows on alt tab) or maybe were copied from some previous one. I don't use docks, I take my time to remove them from the desktops I use. If I can't, I use a different OS.
And way better than anything that involves reaching for the mouse, which would seem to be an essential part of doing it in a 3D DE.
No more buying monitors, just buy one headset and I'm done.
And regrettably it means that now, when COTS hardware components are arguably sufficient for HMDs to start competing with screens, there's no code, no widespread perception of need, and thus no clear market to target. And thus no hardware for sale. Bootstrap deadlock.
Given the effect Minority Report's UI footage seemed to have on popular imagination, perhaps it might be helpful to have some nice demos? So say software devs can have a "I want that!" moment. And industry can have a "oh, well that, that we can build for you now" head-out-of- err, heads-up moment.
Sent from my Lenovo Explorer WMR HMD running on an old laptop's integrated graphics. Simple custom stack. (But only briefly, so I could say that.)
Well, I think that might be a more minor contributor to the problem, but definitely a contributor. The Minority Report interface is utter garbage and I seriously doubt anything we build is going to look like that in the end. Vague gestures that even strong AI is unlikely to be able to correctly determine the target for reliably, arms held in positions that are completely impossible to be held in for any period of time, the entire "interface" is a usability nightmare. Screens far more separated than they should be... just because I can spread my workspace out over 210 degrees doesn't mean that's actually going to be a good idea.
But it's not as cool if you have anything conventional in it.
I'm playing the recent Switch The World Ends With You re-release. It has a mode where you nominally point the controller at the screen to control a cursor. But there's no optical component to it; it just centers when you hit a button and uses the accelerometer. So of course I don't sit there like a dope trying to point at the screen. The controller just sits in my hand where ever it is comfortable, and I twitch in the appropriate manner. It may be "pointed" 60 degrees off the actual screen, but it works fine. I have some similar stories for the Wii, where I still may have had to point the remote at the screen, but I certainly didn't play the games like the commercials. I played them in a much less awesome manner... but a much more comfortable one.
The first Wii Golf was the funniest... eventually I settled on holding the remote pointing straight up, and wiggling it back and then forward again for my stroke. It was not exactly a realistic interpretation of how one swings a golf club. Exquisite control compared to trying to do it the "right" way, though. The second Wii Golf got smart enough that didn't work anymore.
What's a real pity is that there probably is a really cool demo waiting for this. It just involves thinking a bit more creatively.
I agree the Minority Report UI was poor. But seeing it, seemed to get people talking. People who normally wouldn't think about possibilities for future UIs. It seemed to fire imagination, to create anticipation. It had random TV news people talking about UI design. My impression is an "Iron Man" UI was similar. So UI demos, at least in blockbuster movies, can create popular interest.
There currently seems an absence of anticipation for VR/AR "office work" screen-replacement UIs. And there is currently a failure to market VR hardware with tradeoffs made with that focus, rather than for gaming. I was speculating that we might get such hardware sooner, by creating a perception of a market worth attending to, by increasing popular anticipation, by making demos.
Because waiting to get it as a side-effect of gaming tech has already cost a year or few, and seems likely to cost more than a couple more. I'd rather not lose a decade unnecessarily waiting. I can't spare one, and it's not clear society can either.
Regards controllers, oh yes. Remember skeuomorphic design for user interfaces? Things should look like their physical counterparts? A calendar app should have a worn leather border? To ease on-boarding of all the new users not yet trained on phones? I suggest VR is in its skeuomorphic design phase. In contrast, my own interest is in expert UIs for all-day every-day use. So as you say, they are minimum-viable-twitch ergonomics. And any resemblance to the physical world, with its many and unmotivated constraints, is a design smell. But people are focused on gaming, training, CAD, and tours, and Reality is even part of the name. While aphysical UIs seem clearly the way to go, for much non-novice use.
Do you think there would be any interest in a demo gif of a laptop with "mouse ear" mirrors, providing parallax to a laptop webcam with a clip-on fisheye, for high-resolution hand and chopstick tracking?
Doesn't really compete with a regular keyboard+monitor in terms of ergonomics/eye health/fatigue/input methods right now. Maybe in 5 years
I think we may actually be there with the Vive Pro, but that's around $1000 for the HMD + tracking gear alone, and I can imagine that using a keyboard isn't going to be easy, even for touch typists.
> I think we may actually be there with the Vive Pro
Perhaps if ones work environment is green terminals. I once heard rumor of an ops person being happy. 1600px sounds like a lot. But Vive's PenTile subpixel layout means only green is full resolution. And at least with Vive and WMR lenses, there's only a small region a few hundred pixels across where pixels aren't blurred together. Non-PenTile (non-Odyssey) WMR gets you color and subpixel rendering. But most devs will likely be unhappy peering through a VGA portal, even if you minimize head sloshing by remapping it. So basically no, current HMDs are far from competitive with screens.
With better lenses and a larger panel, maybe. But the market seems mistakenly blocked on a misconception of "need immersion -> need high fps -> need next-gen GPU and game/cad support". Not "you can run existing HMDs on old laptop integrated graphics" (I do), and at least some people will buy an HMD with 4K panels, and even use them with laptop low-end integrated graphics, as soon as someone gets around to selling them. Perhaps even with the same blurry lenses. Desktop-ish screen replacement is very different than gaming. Gaming has lots of severe constraints. Which was nice for pushing some tech progress, but now it's causing a stall.
[citation needed]
To be fair this doesn't necessarily mean AR/VR is the future. It could mean a physical designed for computing like Dynamicland or a single device that has intelligent real-time understanding of the physical world that isn't and hmd or a phone.
Any favorite ideas or literature pointers?
I've an interest in coding in VR/AR. But even pushing current HMDs with custom stacks, it's not, for me, yet competitive with screens. Since the hardware market is being so dysfunctionally slow to do the easy, I'm stuck to doing hybrids. Blending of screen 2D, anaglyph, and HMD. Laptop with kludged optical hand and chopstick tracking. Annoying eyewear swapping.
Point is, I've been surprised by how unhelpfully limited the 3D UX literature I've seen has been. It's almost like no one has pursued it with serious intent. That "almost a joke" resonated. Perhaps by the time someone struggles with interface devices, and a bit of code, well, you have your paper or thesis, and there's no big market, so you're done.
But with improving web dev and python ecosystems, it's now more plausible than ever to do full-stack reinvention, devices to environment to apps. No market, but something for itch scratching and joy. Still daunting, but seemingly small-team/silly-person accessible. But I'm missing that warm fuzzy feeling of knowing just how I'm positioned wrt an organized professional research effort. And the collaboration.
Perhaps it is because we're interacting with 3D objects in real space all the time. Unfortunately we're still lacking a convenient and widely available device to interact with virtual objects. VR/AR helmets are not convenient. <sarcasm>Imagine the success of 2D desktops if we had to wear sensor gloves instead of using mice and touchpads</sarcasm>. Even them were not optimal. Touching a screen with a finger was so better (for low precision pointing) that for some people it's the only interaction device with a computing device.
How about Starfire (1993)? This was Bruce Tognazzini's big project. See how many modern UI elements you can pick out of this concept video:
1. Very large form factor concave displays. (We're getting closer, but the HD era really slowed display innovation.)
2. Good gesture input.
3. Seamless integration of drafting/drawing surface and display.
4. Personal application omnipresence across all input devices (personally, I think this is the most significant element often seen in science fiction that is missing from modern computing.)
A current analog is pairing an art display tablet with normal screens. But people arrange them variously, and not-infrequently on separate and movable arms. So seamless physical integration might not be the right thing. Another analog is the new Lenovo Yoga Book C930 (2-in-1 with keyboard replaced by an e-ink touch/pen screen).
[1] https://en.wikipedia.org/wiki/PowerBook_100_series [2] http://oldcomputers.net/ibm-thinkpad.html [3] https://www.mobilephonehistory.co.uk/motorola/motorola_flip_... [4] https://en.wikipedia.org/wiki/IBM_Simon
That page lists some descendants.
After unveiling the prototype, Steve Jobs called Schwartz's office and told him that Apple would sue Sun if they moved forward to commercialize it. Jobs claimed that the project is infringing Apple's IP.[3] Regardless of the threat, Sun determined that the project was not a priority and decided not to put more resource to develop it further into product quality.
Not sure. But around that time Sun had invested heavily in the Gnome project and made a lot of commits that enabled lots of more "boring" things to be added. eg. i18n, accessibility etc. This was because with Linux based Java Desktop (i.e. their polished gnome implementation) they had won various corporate & government contracts in different industries which required these changes, benefiting the gnome project immensely.
Whether the very popular compositing WMs (e.g. Metacity/Compiz) indicated a necessary investment in that area (hacker prelude to new industry), I don't know.
I'm guessing it didn't help though.
I didn't have video or semi-transparency, but I'd gone for a similar idea of clearing windows out of the way - sitting at angles on the sides of the screen.
I realised too late that what I really wanted wasn't 3D, but documents as first class citizens, with modular tools working on documents rather than monolithic applications.
If you clicked on a file icon it would graphically 'open' the file and bring it to the foreground for viewing, then you'd bring tools to it - for example, you'd have application-agnostic text tools for text editing.
I started adding things like 'filters' - where you'd drop a file on a filter icon and it would be animated going through the filter and dropping onto the 'desktop' underneath - e.g. you would be able to apply troff+pic and get a diagram.
It was all naive and probably based on ideas that others were having and I'd seen around (e.g. articles about OS/2), but it was enjoyable discovering more and more ideas as I dug further. My real inspiration was, I believe, TkDesk, which wowed me when I saw I could edit the functionality of my desktop /on the fly/.
What we have in modern desktop environments is great, so I'm a bit sad that the idea of a powerful desktop environment seems to never have taken off as far as some of us hoped was possible. Maybe it just doesn't work!
Anyway, back to the command line...
I think here is the newest version existent: https://github.com/compiz-reloaded
With Novell out of the picture, IIRC the project was forked, rewritten and merged a couple of times. Canonical ended up driving most of the development since. Last time I tried it, a year or two ago, it still had those cracktastic special effects, e.g. the flames circling the pointer when you activate the "Show mouse" shortcut.
Demos are not (largely) interactive, so the demoscene is not known for collective skill designing GUIs.
The most similar/related type of things I can think of that are interactive are crack/keygen programs that include chiptunes and maybe a cute graphic effect somewhere.
For some time I've actually wanted to go track down such programs to analyze/compare/study the progression of design. Kind of difficult to do so though, for obvious reasons (unfortunately).
But my impression from the one or two programs I do very vaguely remember using, half for entertainment purposes was that the UI design was sort of at the "48%-50%" mark - good enough, but flitting just below the halfway mark instead of just above it. Not remarkable; maybe a few rough edges.
Thanks for your comment. It prompted some mental gears to spin and for me to realize the above.
--
I do think I understand the spirit of what you're getting at, though, and there are a few things that my brain has decided is relevant, although I can't describe the working-out process that led me to list these. ¯\_(ツ)_/¯
- SymbOS (https://en.wikipedia.org/wiki/SymbOS): preemptive multitasking, windowed GUI, storage on devices up to 128GB, runs on home computers using the Z80 (starting at 4MHz) with up to (_up to_) 1MB of RAM. (I found this rambling but very interesting demo of what it can do with one of the more capable machines it supports: https://www.youtube.com/watch?v=2-oBNh0UkQc)
- kOS (https://www.youtube.com/watch?v=kTrOg19gzP4): limited-access OS and brutalist GUI using the K functional programming language (hmm, I should watch this video again)
- KolibriOS/MenuetOS: you're likely already familiar with this; it's a pure-assembly-language GUI that runs on 32/64-bit x86 PCs
- AtheOS: Amiga/C64/etc inspired kernel + GUI + application suite, written (for x86) by one person over circa 10 years before it became boring; a small team picked it up a few years later and branded their continuation of the OS as Syllable, although the Syllable website seems to have died at some point
- Contiki: runs a GUI on a Commodore 64 within 30KB of RAM
- There is of course TempleOS, FWIW
It seems since 2006 the world has gone exactly in the opposite direction: simpler UIs based on windows that fill the screen, one at a time or, in some cases, docked. The idea of having multiple windows open on the same screen space was fancy but often confusing for a lot of people. I have relatives who have had their first approach with computers at the age of 80 and can use quite proficiently an Android phone; others that are still struggling, after twenty years, with the concept of "windows (partially) hidden behind other windows" :).
Relevant quote (in the first 20 seconds): https://www.youtube.com/watch?v=Ep_kaY4K5a8
tldr: The revolution happened at Parc in the 70ies.
https://jonathanischwartz.wordpress.com/2010/03/09/good-arti...
Steve didn't, but Jonathan killed the project non the less, because he listend to consultants instead of listenting to devs.
Would have liked to see this thing fly.
Might be helpful in current, resolution-limited, VR.
This is not difficult to do with today's technology. Maybe a similar WM will come along again.
At the same time, speaking from personal experience - 3D in VR can give a much stronger sense of overview of many datasources etc. while at the same time can reduce semantic noise.
I get quite a few productive hours out of a successor to this: https://www.youtube.com/watch?time_continue=1&v=Js7Y1H5D8cY
Because it's useless garbage that nobody needs and that doesn't fix any problems.