How does the Hololens 2 matter?
stevesspace.com
stevesspace.com
FOV is better characterized by a 2-dimensional measurement, like square degrees, or steradians, than a single 1-dimensional measurement, like diagonal angle. "2x improvement" sounds like a perfectly valid way to describe a 2.4x increase in FOV area to me.
> It’s not even double the perimeter
"if we stack 19 Hololens 2 units perfectly, we have ourselves a fully Holographic sphere)...So don’t get hyped on that!"
...and it's rediculous assesments like this (e.g., which have no bearing on a viable AR display) that cause me to stop reading.
I too feel FOV is a key attribute, but for me it's all about it being an important "immersion cue". Objects that we don't first (subcontiously) notice in our peripheral vision, we simply don't believe.
This is actually why I think Magic Leap made a smart decision by making their device so 'enclosed' (as opposed to MSHL that is extremely open and unoccluded), which effectively artificially narrows the peripheral vision. So, by simply blocking off the area of a display window through/from which your device cannot generate image content, they improve immersion. Cheating, but who cares, so long as effect is better than without.
It's focused on having virtual/AR content that is rendered by an AR HMD like the HoloLens, but which is rigidly anchored to a handheld object. So, for example, having extra windows or panels on a phone that float beyond the rectangle that defines the physical smartphone shape.
>This is actually why I think Magic Leap made a smart decision by making their device so 'enclosed' (as opposed to MSHL that is extremely open and unoccluded), which effectively artificially narrows the peripheral vision.
I think this was an issue that the HoloLens 1 had/has in terms of the plastic "eyeglasses" region -- because it looks like it wraps around the full eye FOV when looking at the headset before putting it on, users are disappointed when they wear it and it's revealed just how (relatively) low the FOV is. Setting proper expectations is important for users.
Glad they've revisited those flaws!
As for projecting the fingers, it is possible. It's just not a great experience. Nose-pointer is a (application level) compromise. I've built apps in HL1 where direct hand gestures are essential. And it's definitely a master interface. It's not something you can easily train new users on.
The Go’s screen is appreciably better than the Rift’s, and the network streaming utility that brings your Windows computer’s screen into the Go is excellent.
One thing that was interesting was the zen of using it instead of a monitor - it’s just you and your screen. Nothing else in visible space. With a pair of noise cancelling headphones, I’m convinced that this is the first glimmer of productivity in the commercial space for VR that has wide appeal.
I tried it for a fair amount of time and I’m convinced that when the Oculus Quest comes out, I’ll use that instead of a monitor for some of my work.
There is no one universally great input method.
Too many times to count and I'd disagree it's better than a keyboard equivalent. It's easier for new/casual staff and can be easier for visual people but it's awful compared to someone with a bit of training for complex things. They seem to fail at making the complex things achievable for the less trained as well IME.
Plastering drink logos on big buttons is easy for some, but for non-visual people and those not familiar with the products it's harder/slower than an alphabetical list or typing the first 2 characters. Want to put something on my account? Good luck with the touch keyboard.
Compare those modern touch screen apps to something like a TUI that used to run in the local video shop 30 years ago and there is no comparison in speed and efficiency.
https://www.i-programmer.info/news/99-professional/6263-code...
Anyway, I wonder what happens when VR manufacturers increase FOV to the point where the distortion from the traditional perspective transform (homogeneous linear transform) becomes impractical due to distortion towards the edges (example: https://www.youtube.com/watch?v=ICalcusF_pg).
You need >120 FOV for an immersive experience, but current graphics pipelines are built on the assumption that straight lines in world space map to straight lines in screen space, so you can't do proper curvilinear wide-angle perspective with the existing triangle rasterizer architecture.
[1] https://wiki.panotools.org/Panorama_formats#Cubic [2] https://en.wikipedia.org/wiki/Cave_automatic_virtual_environ...
Isn't that approximation still fine as long as a single triangle doesn't span more than, say, 5 degrees of your FOV? If you are trying to represent a real scene, your triangles will rarely get that big. If you really want to display huge triangle, then you can just sub-divide it into a thousand smaller triangles.
Then you post-process the screen space, with something like a fish-eye shader to warp your image appropriately. Sure you'll lose some resolution near the borders but the human eye won't care because it's not in its high resolution area.
With current GPUs and game engines, you need multiple cameras to achieve such effect, but it has the advantage of drastically lowering the computing demands for high-resolution VR graphics (compare 8k per eye, vs just two blended 1080p views).
You could also ray trace a single view of non-homogeneous resolution that directly takes into account the distortion characteristics of the HMD lenses, but it would likely a lot less efficient than traditional rasterization with multiple blended cameras.
Where does this come from? Is this a reference to research? I find the 110-120 degree FOV is pretty immersive in current day VR.
Does it have to do with: "Objects that we don't first (subconsciously) notice in our peripheral vision, we simply don't believe." (From another thread.)
Tech schools are a thing - and many of them are very good. And apprenticeship has been around for millennia, and is a wonderful way for someone to learn a trade or craft.
- Realize that you should try to research which parts of your breaker box are deadly in the first place
- Find the model number on the breaker box
- Google it, and sift through the results to find relevant information about which parts to touch/not touch
The mental loops you have to jump through would convince most people not to try at all, and just call up a professional. An annotated picture would be just as good, but having the hololens understand what you're looking at and the context of your situation (no doubt a difficult set of problems to solve) would make a huge difference.
I would postulate _exactly_ that the glasses would come with software preinstalled. Similar to how YouTube videos supplement so many instruction manuals today. Even then it's no directed primarily at the person that wants to repair a breaker box but instead the technician that would _install_ it. Though once the form factor is as ubiquitous as cell phones I'd imagine every support service will have an offering. I bet every Verizon tech would love to see exactly what Grandma is looking at when asked to reset her router. Which _then_ us to the opportunity for first responders to assist one another remotely. It's a value multiplier when we reduce time and increase quality in the same swing.
Look at the story about Azure Kinect helping to reduce hospital bed falls from 11,0000 a year to 0. We'll see similar reductions to: House fires do to rushed wiring jobs, defects in nearly any manufacturing process.
Remember that this is 'augmented' reality. What I hope isn't lost on folks is that the thing we're augmenting is ourselves. Granting a super human level of awareness and cognitive ability.
tl;dr - This is Tony Stark level tech.
Maybe AR will be the missing building block that makes high tech manufacturing breach the complexity wall where things start to be getting easier, but without more compelling arguments for that side than I have seen so far I will continue suspect that it will just be the opposite of keeping it simple. (This is a general pair of polar opposites I like to use as a mental model in programming: either KISS or climb the complexity/abstraction mountain until it gets easier again - the problem is that you don't know in advance how high the mountain will be, how low the trough on the other side will be, could be higher, could be lower than the default state of KISS and that it is easy to get lost on the way up)
That said, you could probably wire it up to terminal and split out movable windows with a tmux wrapper. Impress your friends and strangers while you hack the Gibson, then take it off and use your laptop screen because you’re frustrated.
To get what you just called a baby step is an act of barn raising. By thousands of dedicated professionals, tinkerers, artists, scientists, and large corporations. If we could have willed the “baby step” into being before now we would have. Actually many people have, but then quickly notice what’s lacking and get back to raising the barn.
Here are a couple projects I pulled with a quick google. I have a bitbucket wiki with dozens of similar projects, some in AR, VR, projection mapped, single user parallax, with gesture tracking, with custom controllers, etc... This is definitely one of those problems where it's easy to imagine so people imagine it's easy. Maybe we need you to help! Get a spatial display of some sort and lets get to work!
I am a curmudgeonly luddite, obviously, but I would pay thousands of dollars just to have a multi monitor setup that required zero effort and worked on the move (to the extent that I'd be willing to go out with the equivalent of a Segway attached to my head).
If _that's_ still impossible (i.e. it's impossible to accurately and quickly orient the device in space, even in the controlled environment of my desk, or it's just not possible to display text that's nice to read on these devices) then I don't see the point of even attempting the more esoteric stuff as anything but pure research. Clearly several multi-billion dollar companies disagree, so I'm happy if all this comes to pass either way.
Elderly, memory-impaired (dementia, Alzheimer's, etc.), households with children would find immediate use with the assistance, even with not-great accuracy: any kind of reminder that scores greater than zero success at finding misplaced objects would be welcome than the alternative of zero effective recall.
If only the battery life was better, then small business warehouses would find immediate use for these as merchandise locators, as a lot of them have very haphazard methods to lookup merchandise locations.
TLDR: As of right now, your wildest dreams are pretty possible. In the next 2 years, how we compute is going to get strange. Nothing is nearly as simple in an XR environment as a traditional computing one and currently most people don't really want what they'd ask for. Building a simple (useful) XR app makes launching a web product look like assembling a lego set vs. landing on the moon.
---------
Imagenet[0] works just fine for the object identification. You would probably want to use RGBd sensors like the kinect[1] or Intel Realsense[2] instead of regular cameras, but tracking like what the Vive[3] uses could also work. The thing you just proposed would involve a network of server processes handling the spatial data and feeding extracted relevant contextual information to a wireless headset at a pretty crazy rate. Just to give you an idea a SLAM[4] point cloud from a stereo set of cameras or a cloud from a Kinect2 or Realsese produces a stream of data that is about 200mb a second. Google Draco[5] can compress that and help you stream that data at 1/8 the size without any tuning.
Extracting skeletal information from that is really something that only Microsoft has reliably managed to deliver and it's at the core of the Kinect/Hololens products. NuiTrack[6] is the next best thing, but registering a human involves a T-Pose and gets tricky. Definitely you could roll something specific to the application, maybe just put a fiduciary marker[7] or two on a person and extrapolate skeleton from knowing where it is on their shirt. You will also want to be streaming back the RGBd, IMU, hand and skeletal tracking from the headset back to the server. This could help inform and reduce the tracking requirements from the surrounding sensors.
Out of the box, you'd probably need a base i7, 64Gb+ of ram, and a couple GTX 1080s to power 4 sensors in one room. The task of syncing the cameras and orienting[8] them would be something you'd have to solve independently. After having all of that, you would have an amazing lab to reduce the problem further and maybe require less bandwidth, but very probably to get where you're going you'd need to scale that up by 2x for dev headroom and maybe run some sort of cluster operations[9] for management of your GPU processes and pipeline. Keeping everything applicable in memory for transport+processing would be desirable so you'd want to look at something like Apache Arrow[10]. At this point you are on the edge of what is possible at even the best labs at Google, Microsoft, or Apple. The arrow people will gladly welcome you as a contributor! Hope you like data science and vector math, because that's where you live now.
After getting all of this orchestrated, you now have to stream an appropriate networked "game" environment[11] to your application client on the hololens, but congrats! You made a baby step! Battery life is still an issue, but Disney Research has demonstrated full room wireless power[12].
Now all you have to do is figure out all the UI/UX, gesture control, text entry/speech recognition, art assets, textures, models, particle effects, post processing pipelines, spatial audio systems[13], internal APIs, cloud service APIs, and application build pipeline. The Unity asset store has a ton of that stuff, so you don't have to get in the weeds making it yourself but you will probably have to do a big lift on getting your XR/Holodeck cluster processing pipeline to produce the things you want as instantiated game objects.
Once that's done, you literally have a reality warping metaverse overlay platform to help people find their car keys.
What's crazy is that you can probably have all of it for under $15,000 in gear. Getting it to work right is where the prizes are currently living and they are huge prizes.
[0] https://en.wikipedia.org/wiki/ImageNet
[1] https://azure.microsoft.com/en-us/services/kinect-dk/
[2] https://github.com/IntelRealSense/librealsense
[3] https://www.vive.com/us/vive-tracker/
[4] http://webdiis.unizar.es/~jcivera/papers/pire_etal_iros15.pd...
[5] https://github.com/google/draco
[7] https://www.youtube.com/watch?v=JzlsvFN_5HI (markers are on the boxes not the robot)
[8] https://realsense.intel.com/wp-content/uploads/sites/63/Mult...
[9] https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus...
[10] https://arrow.apache.org/
[11] https://docs.unity3d.com/Manual/UNetClientServer.html
A nice VR gaming experience is hard. Immersive means you can't see the real world, so your balance rides on rendering, requiring high fps, low latency, and thus lots of gpu. Means a high bar for avoiding "immersion breaking" hardware and software visual oddities.
The perceived near-term market for VR gaming is larger than that of VR or AR desktops, so that set of hard has gotten investment, and the other, not so much.
Market structure, especially patents, discourages commercial exploration of smaller markets. Say you want to build and sell your great system, and that it happens to need eye tracking. One eye tracking provider sells very expensive systems to industry and government, and your volumes are far too small to interest them in cheaper. Its competitors have been bought by bigcos, and no longer offer you product. Patents, and your small market, block new competitors. So eye tracking is not available to you or your envisioned market.
Community communication infrastructure is poor. Suppose all the pieces exist somewhere. People interested in desktop experiences, willing to throw a few thousand dollars at panels, tethered gpus, eye tracking, and so on, willing to be uncomfortable, to look weird, to tolerate a regression in display quality; a 2K HMD with nice lenses; a similar 4K panel; the electronics to swap the panels; some tracking solution; and so on... Even if all those pieces exist, the communication infrastructure doesn't exist to pull them all together. Neither as forums, nor as markets.
The poor communication degrades professional understanding of the problem space. People are unclear on the dependency chains behind conventional wisdom. So high resolution is said to require eye tracking or high-end gpus. VR is said to require high frame rates. And the implicit assumption of immersive gaming is forgotten. I've found it ironic to read such, while in VR, running on an old laptop's integrated graphics, at 30 fps. Wishing someone offered an 4K panel conversion kit, so a few more pixels escaped blurry lenses. A panel I could still run on integrated graphics, albeit newer. So part of the failing to pull together opportunities, is failing to even see them.
Perhaps in an alternate universe, all patents are FRAND, all markets have an innovative churn of small businesses, and online forums implement the the many things we know how to do better, but don't. And there's been screen-comparable VR for years. But that's not where we're at.
10 years ago there was a booming scene of open source natural user interface projects. People building huge multitouch interfaces, experimenting, releasing open SLAM tools, and exploring what you could do with DIY AR/VR/projection maping/natural feature tracking/gesture interfaces. Post the success of the first Oculus dev release all of the forums went quiet, the git repos started being unmaintained and outright scrubbed. The main contributors to the community got scooped up by the motherships and any supporting technologies locked down in what felt like about six months to a year. Leap Motion was a standout company from that time. They have been selling the exact product they built then with almost no improvement until recently. Somehow they weathered the storm, didn't sell, and are doing some really neat stuff now. Structure IO took up the stewardship of OpenNI and if you look hard enough you can still find cross platform installers that have the banned original kinect tech that Apple bought and is extremely litigious about keeping off the internet.
Hololens is the Vive/Rift of AR and Vuzix Blade is the Focus/Quest of AR.
No end user will buy a Hololens if it looks like that and costs thousands.
But something like google glass in good looking, with more power and good voice control could really replace Smartphone in the future.
Still the Hololens will sell to enterprise customers and AR play-rooms where you can use them for a small fee for an hour (like we have VR rooms right now).
So I don't expect much more from new Hololens generations than evolutionary improvements.
It looks like the author got confused and multiplied the wrong numbers. 43 x 29 = 1247, so this is about a 2.4x increase in area.
Using this solid angle calculator to compute the solid angle covered by a rectangle [1], I get that a 43° x 29° rectangle subtends 0.335 steradian (sr), while a 30° x 17.5° rectangle subtends 0.151 sr, making for a 2.2x increase in solid angle.
However, the numbers 43° and 29° apparently come from applying Pythagoras to the 52° diagonal field-of-view (fov). That's also incorrect, and I haven't done the math to correct it. (As an extreme case, for example, a 180° diagonal fov gives 180° horizontal and 180° vertical fov, so Pythagoras clearly breaks down.)
> We need a search engine for holographic layer services. It would be like Google, but for MR experiences. Holographic services would use a protocol that defines a geofence to be discovered by the layer search engine's crawler over the internet (this could just be a meta tag on a classic website). The HoloLens or whatever MR device would continuously ping the search engine with its location, and the results would be ordered based on their relevance (size of geofence and proximity are good indicators). The MR device would then show the most relevant available layer in the corner of the FOV. Selecting the layer would allow enabling it either once or always, and the device would then deliver the holographic layer over the internet. The holographic layer would behave like a web service worker (in fact, it could be a web service worker) and would augment a shared experience which contains other active holographic layers. For example, your Google Maps holographic layer could be providing you with a path to walk to the nearest Starbucks, and once you're outside Starbucks, the Starbucks layer is also activated, which allows you to place an order.
> This concept of activated layers, I think, is a great way to avoid a future where we're being bombarded with augmented signage and unwanted experiences. In fact, you could go further and enable blocking notifications about specific/certain types of available services. (ie. don't notify me about bars or fast food restaurants.)
I also think this could have applications within intranet environments in corporate/enterprise contexts: several teams could each develop their own layers used for different purposes. That would make something like this worth while to pursue today, seeing as HL2 is solely targeting those environments for now.
So I guess for a long time we will stuck with one app a time, and some kind of manual switching.
Also there is privacy angle, I don't think nobody wants pinging their location all the time, maybe some QR code/ beacon solutions can help, but I am not much optimistic there too.
Maybe you could have something like K-anonymity? You discretize your location into chunks of maybe 100x100 m, take the hash of the chunk ID, and request results for all chunks whose hash starts with the same 4 or 5 digits as your chunk's hash.
Would probably require some more thinking since a consecutive series of requests for adjacent chunks could be used to uncover which particular chunks you were interested in.
For instance, it’s always more efficient to intervene/support a student in the moment they are having trouble/ready for the next idea, but there’s a natural limit to how much information about a class full of students is available/understandable; better tools for allowing me to know how my students are engaging/progressing/struggling/succeeding while they do the work would be wonderful.
My main use case would be to free me from sitting by a screen. I want to be able to access mainly Slack, a browser and some IDEs so I can casually do some code review or chat to team members as I do some chores around the house. I'll need notifications as well. But mainly I want to be able to read stuff while being mobile.
Are there AR goggles that will let me do that?
What makes them think there is a market for these beyond gaming and game like scenarios like military ?