A developer's view of Vision Pro
david-smith.org
david-smith.org
I left Apple 1.5 years ago but was working on the Vision Pro while I was there. I have spend many hours working in the headset and I know exactly the feeling he's describing! Leaving felt like going back in time to using clunky technology and I've been waiting for outside world to catch up (and will still be waiting until it comes out at least).
For the past week I've been trying to explain to people how certain I am that doing work in headsets will become mainstream but am (understandably) met with doubt, and I think you'd need to try it on to fully understand.
Here's the one that I bought to use with my "frankenquest" (Vive Deluxe Audio Strap on a Quest 2) setup: https://www.studioformcreative.com/product-page/vive-das-200...
I'd say that eye strain wasn't really an issue at all for me. When you take off the headset after a while inside of it you kinda get a jarring transition back to normal vision instead of passthrough, but passthrough doesn't have any noticeable issues when you're inside of it. It's almost like waking up from a dream where stuff feels different and you can't exactly place why.
As for physical weight, it's lighter than other VR headsets I'm used to but obviously more than a pair of ski goggles. I'll echo MKBHD's thoughts that the most noticeable strain was on my nose because my lightseal (like the WWDC demos) wasn't as personalized as the version you'd get as a customer, so I fully expect that to be a non-issue.
Also, is passthrough good enough for things like typing, so you don't get an uncanny valley sense of the keys being just slightly off visually compared to your proprioception & touch?
I don't really look at my keyboard while typing so that's also a non-issue. For stuff where you really need hand-eye coordination it's totally fine at normal arms-length but gets worse as you get closer to your face. Finding a key on the keyboard with your eyes is not a big deal but if you try to drink from a glass of water you'll likely spill it on yourself the first time (and then eventually get used to it).
I wondered about that.
Great to hear about heat. That was always an issue for me in other headsets.
Either way, very fair.
I'm not sure how common this is for neurotypicals with VR headsets. With Vision Pro have you experienced losing track of time? Or does the AR passthrough seem to help with this?
I'm not sure if that's due to the passthrough or the fact that I was only ever using it to actively do work (and I usually mean work on stuff for the Vision Pro itself, not just "normal" dev work).
So many questions I’d love to ask you about this, but again, none of my business. Just wanted to let you know I’d upvote that blog post to the moon should you ever decide to write it.
Feel free to ask any questions here and I'll answer anything I can (e.g. no unreleased info, but maybe go into more detail about the kind of stuff that was shown to devs at WWDC). No blog right now but maybe in the future :)
The idea of looking at real screens in passthrough didn't even occur to me.
If you're wondering about passthrough it's pretty good for most things but is definitely missing the dynamic range of the human eye, which no video camera or display can really match yet. Like a super bright light might just show up as white and you can look directly at it no problem. Basically the same as what you get when you record a video and watch it back on your phone.
I'm sure as soon as it's released people will be doing that and reporting back though!
I do think a bunch of incremental improvements will eventually allow different use cases though. E.g. you'd never take it on a run or a bike ride in its current state but with enough weight and battery improvements you could have a HUD for exercise stats + navigation when biking with stuff like a videogame-esque ghost of your PR to race against. Just a random idea, there's a lot of stuff you can dream up.
Imagine riding a bike at 30kmph and accidentally unplugging the headset. Boom. Black. Instantly. That's a scary place to find yourself.
I'm curious if Apple will utilize the gyroscope to try and detect movement faster than a certain speed and show a warning. Take the risk if you want, but at least people should be aware of the consequences!
> I'm curious if Apple will utilize the gyroscope to try and detect movement faster than a certain speed and show a warning.
The Vision Pro is doing SLAM at all times so it actually always knows exactly where you are (relative to its origin) and your speed!
2. What about sleeping with it? VRC people do that, so they might be curious.
3. Is it usable outside?
4. What was the longest time you had it continuously on?
5. Have you ever dropped it? :D
2. Never tried sleeping with it... I don't see why it would be any worse than other headsets though.
3. I've never used it outside, but that was for secrecy and not technical reasons.
4. Honestly not sure, maybe an hour without taking it off at all but I've definitely been in it for the majority of a few hour spans many times. At the time the main blocker was the beta OS and not comfort or battery (I would keep the battery pack plugged into the charger most of the time).
5. Nope! We were all super careful with them because prototypes are expensive, much more so than the consumer product. It's not something you could just casually drop while using like your phone though.
4. So the battery can be charged while plugged into the headset. What happens when you pull the battery from headset? I am guessing insta black. Does it have some power-saving mode where only R1 is feeding images from cameras to displays without any computing possibilities?
This sounds like the battery can be charging while using the headset, right? Which imo makes the 2 hour battery life much more understandable – if you're stationary in the device most of the time then you only have to rely on the battery when you move. If you can plug in to charge when you're back at your desk/couch, it's not really a limiting factor (for the use cases Apple is pursuing).
https://daringfireball.net/2023/06/first_impressions_of_visi...
What does "border in the FoV" means?
Either one of those statements can be right. Not both at the same time.
It's time someone with real measuring equipment looks at of these and gives a more technical review than just "wow Apple magic".
meaning a couple 4k screens can be rendered in the visible area without a noticeable difference in quality to real screens at a similar "apparent" distance
We also know 23 million pixels for the whole system. So, 11.5 million for one display. a 4K display is 8.2 million. So it's not a whole lot more than 4K, and lower than their own 5K display (which has 14 million), which matches what they claim ("more than a 4K display"). That's not enough to cover the full field of human vision and still have pixels so small they can't be seen.
With corner waste it's just barely enough for one 4K display this way (and stretched to the full limits of vision it will be pretty unwatchable so close). You can't display two as you mention, because AR/VR projection works by using the 2 screens to display the same content just from a slightly different position (parallax) causing the 3D effect. The more overlap between the eyes the better and more comfortable the 3D effect (some headsets try to get an ultrawide FoV this way but shoot themselves in the foot with low overlap).
If it has a really wide FoV its sharpness will be pretty much on-par with a quest 3/Pico 4, if it's got the same FoV it will be a lot sharper. I expect the truth to be somewhere in between. Wider than a Quest and also sharper, but pixels visible if you look well and not quite full human FoV.
What I expect Apple will have done is sacrifice a bit of vertical FoV for horizontal FoV. Vertical FoV is important in VR (especially 'roomscale') because of orientation issues, and not quite as important in AR. Also most of their marketing material promotes a seated position so moving around is not an issue. Most VR headsets have an almost-square resolution per eye, but I expect this one to be closer to 16:10 (maybe not quite that wide). I think it will come out at around 4300x2600 pixels per eye which is slightly over 11 million pixels.
Still very impressive (which it must be at that price point obviously!). But real-world and not magic. Makes sense because Apple's engineers, as good as they are, are bound by the same laws of physics we all are. Their marketeers would love us to believe otherwise though.
It's really time for some real-world specs on this thing instead of marketing blah. But I think Apple specifically doesn't want this, it's no wonder they let only the most loyal media outlets (like Gruber) get as much as a short hands-on with this thing. And even those experience are white-gloved in detail (they even prepared custom adaptive lenses). They just want to keep the marketing buzz on as long as possible.
The exception to this is at the edges of your vision, where each eye does see a unique portion of the field, but by definition that’s not where you’re looking.
This is impossible at the resolution stated. The headset would have to be 8K or more per eye in order to achieve this, which it most definitely is not.
I'm not so sure. Human vision is only sharp in a coin-sized area at any time. If you fix your eyes on a single word in your comment you can't actually read the entire comment, for example.
In other words, you don't need 4k in your entire FOV you just need to ensure that most of the pixels are spent in the middle of the viewing area.
I believe that it would be possible to have the screens in the headset generate a very distorted image, where the edges are compressed to a small area of the actual screen and therefore low-res, while the lenses stretch this image to fill the viewing FOV. Kind of like anamorphic movie lenses, or even how wideangle lenses distort the edges more than the center.
I have no idea if Vision Pro does this but it seems theoretically possible at least.
I think what the OP means is that you'd have actually less physical resolution in the sides. Most headsets do this somewhat but not in a huge amount.
tf? Did he miss everything about LLMs?
This is actually a bit disappointing since I would rather not have to move my entire head to look at a virtual side monitor. It seems like the technology is there now and many companies are looking into it, hopefully other headsets will be released with higher FOVs: https://youtu.be/y054OEP3qck?t=283
Another noteworthy point from that video: Apple has bought Limbak, an optics specialist that was formerly tied with Lynx R1's optics. This means Lynx can no longer use Limbak's future optics that, admittedly, fall short in scaling to higher resolutions. Now, Lynx has shifted its gaze to hypervision optics, intent on preventing a similar acquisition by another tech behemoth like Apple.
I was already into performance programming beforehand but had no finance knowledge whatsoever.
> That shift is fundamental. The interface for Vision Pro felt like it was reading my thoughts rather than responding to my inputs. Its infinite, pixel perfect canvas also felt inherently different. I wasn’t constrained by my physical setup, instead my setup was whatever I thought would be most productive for me.
That DOES seem like a paradigm shift in the offing to me. Sure it might be iPhone 1 expensive and uncertain, but it's easy to imagine how incredible a lighter, more affordable version will be to MANY people within 5 years or so.
Is it possible to use Vision Pro to build and publish a Vision Pro app?
Because it seems to be another Apple device separating producers and consumers just like iPad and iPhone.
Are we going to be able to create a large software project using it without carrying around a MacBook as well?
It’d be nice to be able to run macOS when I travel and not have to bring my MacBook.
I suspect that, like the iPad, there will be ways to force it to function as a standalone device, but that the happy path will be heavily optimized towards you buying as much Apple hardware as possible.
I suspect the downvotes are because the keynote literally featured someone using their Mac inside the headset. And another video (session? State of the union?) showed developing a Vision Pro app on the Mac and testing it right next to the Mac from inside the Vision Pro.
That doesn’t mean Apple will expose that. Maybe just not today. The original iPhone had a full OS capable of multitasking but it wasn’t a feature given to the user due to memory and battery constraints. We’ve now had it for years.
This was the first we’ve seen of it, and I suspect the on device development story was not one they wanted to spend limited time on. Plus knowing Apple it seems likely they’d want to make Xcode work better for the interface than just plopping the Mac app in.
Time will tell. The iPad can do development. Apple has proven it with Swift Playgrounds. But they still haven’t given us Xcode. So who knows.
Or are you speaking instead to a local terminal environment, like iSH or a-Shell on iPad?
Apple announced just before WWDC the iPadOS versions of Final Cut Pro and Logic Pro, their professional media creation tools. We've all seen shows and listed to music (and podcasts) created with these tools. While most of the core functionality is the same between the Mac and iPadOS versions, the iPad versions take advantage of multitouch and other iPadOS features, as they should.
> Plus knowing Apple it seems likely they’d want to make Xcode work better for the interface than just plopping the Mac app in.
Yes; that's how Apple does things.
As I mentioned elsewhere in this thread, iPad users can already use Swift Playgrounds to code and submit an app to the App Store [1].
It's just a matter of time before there will be a version of Xcode for the iPad.
Thats nothing compared to the bash build stages and custom permission setups, third party libraries, bespoke source code management scripts, ruby-based project initialization and preprocessing scripts I've seen amongst Xcode-based projects.
When those fail, you are troubleshooting through logs and at the filesystem level.
If the idea is a version of Xcode without those things but is otherwise fully featured, I think that is actually speaking about some future version of Swift Playgrounds.
Otherwise, "Xcode for iPad" is basically a jailed Mac VM.
basically is it a laptop (fully independent), an ipad (excellent secondary device that can do focused work on it's own), or a smart monitor (peripheral with some built in usability)?
It's using local Wifi (or USB), so the latency is very low.
Beyond that, even though an iPad can act as another monitor, it's not a big monitor. It doesn't really achieve the ideal of a workstation replacement.
The persistent framing of i-devices this way on HN, after all these years, is baffling to me. Tons of creation happens on them, or using them as a tool in service of creation, much of which would be harder or impossible with traditional computer. Just not a lot of software creation.
Even in digital art where the iPad excels doing something like having a reference photo open while you work is compromised and the only good way to do is to hope the app you're using supports it a reasonable way or else throw away half your screen real estate.
For most creation-oriented tasks I do that aren't computer-centric, I'd much rather have an iPhone or iPad than a laptop. They're great tools for creation & work, just not so much for creating stuff for computers, certain exceptions aside, and for those they're mostly just about as good as a laptop, or even not-very-good but usable in a pinch, not better.
Desktop shines when you create something novel.
It's as simple as that -- there are tons of things people have created really polished iOS "happy path" software to create on, but they're not general purpose creators.
Specifically because anything general purpose threatens Apple's App Store toll gate, and thus cannot be allowed to exist.
If apple would just get over themselves and allow sideloading (or, gosh, a terminal!), it would go a long way to improving iOS usability for creatives.
…as long as they’re sanctioned by the platform owners.
The main one for me is the longevity of your data. It is connected to the control over your data you have in a desktop OS. Good luck finding what you did five years ago in an app in some device and reproducing it now. You are at the mercy of some proprietary format the app creator chose, the availability of the app at a later stage, whether you saved your work in a cloud service, whether you kept your old device running, the operating system updates you installed, .....
With a desktop you can archive your work, whatever it might be, in sequences of bits that reside in your disk. Open source software and virtual machines go a long way for helping reproducibility.
Not all tasks benefit from reproducibility and longevity but I'd argue they include a lot more than computer programming
Compiling code isn't impossible; every webpage that contains JavaScript uses the Just In Time compiler to execute JavaScript. The ability for 3rd party apps to compile code is limited due to security reasons—Apple's Lockdown Mode [1] specifically disables the JIT.
iPad users have been able to write code using Swift and submit apps to the App Store since late 2021. [2]
I have no doubt you'll be able to create apps using the Vision Pro in the future… there's probably a version of Swift Playgrounds [3] running on a prototype Apple Vision Pro right now.
[1]: https://www.wired.com/story/how-to-use-lockdown-mode-ios-16/
[2]: https://www.cnet.com/tech/computing/apples-swift-playgrounds...
I think the disconnect is in the type of creation, the higher-end production quality you want, rarer iOS devices are. And the corollary is, if you only care about the highest-end of creation, they essentially become invisible.
For producing stuff for computers or for industrial design (e.g. CAD or architectural drafting), yes. In other areas, tablets and phones are ubiquitous tools, and if they vanished tomorrow they wouldn't be replaced with e.g. laptops—they'd be replaced with paper, with various single-purpose devices, et c.
iPad is not because it cannot be used to create the Vision Pro. It may still have advantages where it's better for some uses cases, but it still doesn't make it a general purpose computing device in the same way that a TV is better for watching videos than a Macbook, but a TV is not a "real computer".
I think the real/not-real computer thing's not especially illuminating, but I don't take issue with it the same way I do dismissing phones and tablets as "consumption-only", a perspective I find simply baffling, because I see them used for creation and productive purposes all the time in a ton of contexts by ordinary people.
- Sensors: It looks like the XREAL AIR doesn't have any sensors so just displays the monitor in a fixed position on screen, which is pretty different than loading up monitors in a fixed point in space and being able to look away at other stuff. Also eye tracking is a pretty killer feature that totally changes the game in my opinion, I wouldn't personally buy a headset without it after having experienced it.
- VR vs AR display: As I'm sure you're aware, with AR sunglasses you can't really have anything darker than the actual light passing through so you're limited to only rendering stuff brighter than the surroundings. Maybe for text it's fine but not for images or video in my opinion. I can't find any footage taken through the XREAL AIR lens (and their website doesn't faithfully show this effect) so idk exactly how big the difference will be for you.
- FOV: I'm not actually sure what the FOV is of the Vision Pro but it's significantly higher than 46°. And because usable area increases with the square, you'd be able to fit a few times more displays and other stuff in.
- Resolution: XREAL AIR looks to be 1080p, Vision Pro is way better than that (per-degree too), so you can have better looking and smaller text.
My gut feeling is Quest 3 will be more than good enough for most people/use cases, and people will be more than happy to pay 7x less for what is - at the end of the day - basically the same experience.
Vision Pro looks like a better version of HoloLens of yesteryear. Doesn’t really seem all that groundbreaking to me, just more refined.
I seriously doubt it comes close to what Apple is doing. The price just isn’t anywhere near high enough.
Fact is there is a laptop chip stuffed into this headset, there is a mobile soc in the oculus line.
Although I do think that facebook has sold a lot of their devices at a loss?
Apple loves their profit margins. But it’s not $3500 for fun. It’s clearly expensive to make, even if their profit margin in still baked in.
This product is inherently experimental. They are not going to be operating at the type of scale they do when it comes to iphones, macbooks, ipads, etc ...
So I do imagine that if this thing takes off and they can start putting out hundreds of thousands or millions of units they will gain more savings through scale.
I'd love to know what their initial order for launch is. It will be interesting to see how far the preorder precedes delivery as well.
Full color passthrough from cameras and a depth sensor (better than Quest Pro), 120Hz, slightly higher resolution than Quest 2, 40% smaller (whatever that means), Snapdragon XR2 Gen2 (they claim more than twice as fast as Quest 2), pancake lenses, controllers without the tracking rings, $500.
Compared to Apple's, the main downsides are lower resolution display (also not OLED), slower, no eye tracking.
Main advantages are price, support for games/controllers (you can still use hand tracking if you just want to watch movies), and no external battery pack.
The main difference is that Vision has a laptop grade processor (Apple M chip), LiDAR, and another separate processor to manage the LiDAR, a separate depth sensor, & all the other 12 sensors and cameras
The Quest only has a smartphone processor. The pro doesn’t even have a depth sensor. While the q3 will have a depth sensor, it will not have eye tracking or face tracking, in addition to only running yet another smartphone processor and only ~5 sensors and cameras
I would say 6500 is too much for a better version of something I already have that doesn't really enable any additional usecases. The Vision Pro would be used pretty differently than my Vive (which is just the occasional gaming session right now). I'd think of it as more competing with iPads, external monitors, and laptops than a Varjo right now.
Gaming, TV, casual internet browsing, stationary exercise, socializing with people remotely (e.g. VRChat instead of Zoom/FaceTime), passing time on trains+planes, etc.
Tech enthusiasts underestimate how much the general public hates having gear strapped to their faces and over their hair, especially more than half of women who put considerable effort into their face makeup and hair.
This can
>8k per eye
Not required to get 4K fidelity… you only need somewhat higher than 4K resolution.
This is, for most people, pointless, as the rendering costs will be much, much higher (>8x for a quadrupling in resolution per eye), they probably won't care if text is not quite as sharp as a real 4K display, and they might not even be able to tell if their eyes haven't been trained on them.
Unfortunately though, I require a 4K display for sharp text (I get really weird visual effects from low-resolution displays), and my current HMD (HP Reverb G2 at 2160x2160 per eye) is uncomfortably blurry, so I don't imagine the Vision Pro at an estimated 3400 pixels per eye according to their marketing figure (`sqrt(23000000 / 2)`) can do much better.
I know it's revolutionary and this is about at the current limits of HMD technology, I'm not at all claiming that it's not impressive for the industry. It's just part of the reason why I don't use HMDs often is because the tech hasn't yet advanced enough to be comfortable for me.
As for development of non-VR stuff I personally didn't because I already had a fully-featured desk setup and the OS at the time was buggy enough to not want to deal with it (as every OS is years before release, not a ding on Vision Pro at all). With a good OS (which it will be upon release) and more native apps (like Termius on iPad) I definitely would though!
FWIW the iPad supports dumbed down app development through Swift Playgrounds and presumably the Vision Pro will at the very least support that if it supports all iPad apps, but that's pure speculation.
This gets at my main question about this: does it feel to you like it will be great for "doing work", which generally happens at discrete places, or something which will apply to life generally, which is the kind of (some would say dystopian) ideal of "augmented reality"?
Or put another way, I felt a huge increase in immersion and interest going from the original Oculus's three degrees of freedom to the "room scale" six degrees of freedom in Quest.
I know this is "spatial computing" and you can put monitors in fixed positions in the environment, but is "desk scale" where you can kind of look around and pivot your head, or "room scale" where you can get up and move around the room, or "life scale" where you can leave the room entirely to check on the screens you left in the kitchen before coming back to sit down in front of the ones in your office?
according to the videos they show, people move around a lot in a room with these things on, grab things (e.g. their phone, cup of coffee). and makes sense, if you see basically what you have in front of you, then there's no "boundaries" you should stay in not to bump into things.
I also don't expect it to be super dystopian or replace real life in general, but it could definitely replace the parts where you're already looking at a screen regardless.
It is technically capable of being "life scale" in the way that you describe: it tries to keep a consistent coordinate system across multiple reboots and from room to room and real life objects can occlude virtual objects, but to be honest I have no clue what the average use case will be or how the OS will enforce it.
To me its a bit like tap to pay vs magnetic credit cards. Eventually tap to pay replaces all other ways of paying much like AR might replace all other screen interactions. But FIRST it should replace something critical. In the payment space this was subway entrances where people needed to pay quick and get through.
The reason it takes so long for full adoption is because its very challenging to replace something like magnetic credit card that “works pretty well”. From my point of view, the iPhone “works pretty well” and its going to towards the END of the innovation cycle when other screen usage gets folded into these spatial platforms
The most important part of a WFH setup is desk + chair. That's where the comfort is, much more so than a big screen - which is certainly nice.
Desk + chair is what takes up space though. Once that space is taken, a big monitor is just something you put on it.
As excited I am about this new tech, which looks fantastic from the ad, I don't think it will replace a comfortable desk + chair setup.
At this point, I doubt they’d be letting them off campus.
When you worked with Vision Pro, did it have a device native terminal and/or a device native ssh client?
What I mean by that is a terminal running under VisionOS, as opposed to getting terminal access by first connecting to a remote computer.
I'm sure some developers will switch over on the first generation, some will wait until it gets lighter/cheaper/more immersive (disclosure: I have no clue what's coming in the future), and some might never switch if they just plain don't like having something on their head.
https://github.com/tikimcfee/LookAtThat
AR VR for iOS and macOS. Millions of glyphs. Instant control. There’s magic here. If this excites you, work with me and help make this a reality! I don’t have all of it in me.
I wish I did. I don’t. I don’t have all the time and energy. But there are people here that if they spent just a little time to work on this, we would be in the future of a 3D code space in days, and not weeks or months.
I owe a new readme for the project. If any of this makes you feel any feelies, get in contact with me star it, make noise, whatever!
Lotta love to yall. Thanks for letting me vomit words.
The real question is, can you make two devices sync to each other simultaneously.
Probably not I'm imagining
Third party developers will definitely be capable of adding a sync feature like you're describing but I have no clue if they will.
Within an app, the challenge would be identifying they occupy the same space and mapping into the same coordinate system. Easier for VR mode than AR mode, where your mapping can be pretty arbitrary to the available bounds.
Absolutely. It's impressive work on Meta's side that the full ecosystem of headsets natively support this in AR mode already [0]. It will be interesting to see if Apple has this figured out, or if they can ship in time for release. The fact they didn't demo it suggests to me that they are still in catchup mode here. But then, there are a lot of things they didn't demo that came out in the technical talks later.
[0] https://developer.oculus.com/blog/build-local-multiplayer-ex...
Then the real world will feel like paradise.
It's also not novel to me because I've spent a while using it already, so I'm not exactly in a rush to test it out just to see all the features.
However most of the previews I read do point to some amount of weight/comfort ergonomics that I think play against the productivity use case.
Basically my work compute needs to be ergonomic enough to use for 8-12 hours/day.
I have a hard time believing we can live in this headset for anything north of 2-3 hours at best.
I have to wear a cpap so I haven’t found an eyemask that works with my cpap headgear.
1. Could I take a disconnected cable keyboard and "anchor" the virtual keyboard to overlay the real one to get the haptical feedback when typing on the virtual keyboard?
2. I'm very excited about "hanging out in VR" but initial impressions of Persona avatars seem to be mixed. Do you think people will eventually get used to Personas after using them for longer or are we not quite there yet?
3. What is the focal distance for the eyes?
4. What are you most excited for using the Vision Pro again once it comes out :)?
I definitely can't wait to upgrade from my original HTC Vive!
2. Honestly no clue how the public will react to Apple's avatars, but keep in mind that third party apps are free to create experiences like that and I'm sure you'll see a dozen VRChat clones taking off.
3. Not sure! I never ran into any issues but I also don't need glasses.
4. I personally am a VR gaming enthusiast and despite gaming not being the core use case of the Vision Pro, I'm very excited to see what people do with it. If you've played Half Life Alyx you know how fun gravity gloves are, so imagine having games that have graphics and physics that can seamlessly blend with real life. Use your hands to toss around virtual objects that bounce off walls in the real environment! The cool thing is that this is all super easy to develop for now: all the scene understanding, rendering, hand tracking, physics, etc. is all handled by the device so you can probably whip up a gravity gloves prototype in a few lines of code.
Apart from that, I really hope Apple will work with more game engines in the future. I rented an HTC Vive Pro 2 last year just to play through Half Life Alyx and everything about this experience was so, so good (especially Jeff). Would be incredible if it ran on the Vision Pro eventually but I guess Source 2 is probably not on the engine list :)
One last question I actually had if you don't mind. How do you think gestures will evolve once the headset releases? I understand there are a few basic gestures defined already, like clicking, dragging, zooming, but what about cmd + Z or deleting something. I'm sure eventually people will expect the same gestures for those actions across apps but I didn't see any mention on them so far. Any thoughts on how those might emerge?
The author got to play with the thing, and is very excited to develop software for it.
Altogether sounded like an GPT Apple Ad.
That said I’ve planned to get one since I heard rumors years ago that this was in the works.
I sort of expected it to be a full replacement for a MacBook though.
As for developing from day one, web development should be ok, as things like vs code online have really opened ip options.
> That intuition is developed by following a platform’s development from its early stages. You have to have seen and experienced all the attempts and missteps along the way to know where the next logical step is. Waiting until a platform is mature and then starting to work on it then will let you skip all the messy parts in the middle, but also leave you with only answers to the “what” questions, not so much the “why” questions.
I definitely think there's a lot of truth in this, not just for the Vision Pro but for many (if not most) platforms/frameworks/what have you. This could be an entire blog post in itself.
What did you want them to write about instead?
He may be best known for Pedometer++ and Widgetsmith which was the app that went viral on TikTok when widgets first came out for the iPhone.
Do you think Apple is unaware that he has a popular blog and is a popular developer of software that runs on Apple products?
It’s a developer conference. Yes there are media there, but it’s also to court developers.
Looks like I could use that new AI autocorrect. :)
Excitement is good. It's not a revolutionary take on anything, but it can be interesting to know why a seasoned iOS developer finds this new platform worthwhile to invest in.
developer gets frustrated as Apple refuses to relinquish full control over the device.
I had to use it for a while when I was unable to touch a keyboard or mouse while recovering from RSI and I was surprised by how quickly I was able to get to about 80% of my previous productivity using just my voice. I still use it sometimes even though my RSI is fully healed.
Asking me to "blink twice" or anything like that is going to make my eyes lose focus on the screen/content
Maybe they could add other features from just your face though for other platforms, wiggling nose, raise eyebrows, stick out tong, blow a raspberry?
That or they add voice commands, "open", "select" and such.
I suppose ultimately in spatial computing with voice interface and eye tracking the concept of a "click" may die.
I'd love to have an eye tracking setup though on a normal desktop computer where I could devote a keyboard key to clicking and I'd never have my hands leave the keyboard.
I can cut vegetables without looking at them. I can use that dynamic to offset the planning and acting phases of my thought process. Falling short of that efficiency will feel limiting.
Maybe X is a button on the keyboard. Maybe X is a gesture.
I can think of some Portal puzzles in particular where timing is important, and you need to hold your aim but wait to click until something happens somewhere else on the screen (so the place you're clicking is not the same as the place you're looking).
I think the same thing applies to e.g. recording network activity in Chrome dev tools. My eyes are on the page to see when the thing I'm interested in finishes loading; my mouse cursor is on the button to stop recording.
It's not a super common pattern, but probably common enough that it would be annoying not to be able to do it.
I am speaking mostly about the desktop interactions. In your Chrome Dev situation, I would look at the cursor before clicking on the stop recording button. I think I might be able to trust the MBP trackpad to do a primed click without looking at the cursor, but I wouldn't trust a traditional desktop mouse to have stayed steady enough.
Oh wait some version of that is built into accessibility on mac already (eye tracking mouse): https://support.apple.com/lv-lv/guide/mac-help/mchl437b47b0/...
The mouse is the superior input device. When people who actually need a better input device get one, they get more advanced mice:
I've used a 3D mouse for CAD but am not sure where else it would be helpful?
Nobody wants it for day to day computer interaction. Most people using eye tracking for computer interaction are disabled, because it's a terrible experience.
We can see this with voice to text, in which despite in theory being so much faster than typing things out, tends to not be due to these details (processing lag, clunkiness of handling different forms such as whether a word is part of a command or should be added to the text).
Same will happen here. People will get over the hype period and then realize hey this isn't actually faster or more efficient than a traditional tool. Apple knows this as well, it's why they're marketing it as a media consumption device first and foremost, where a lot of these problems can be safely ignored.
But the screens on Vision Pro are larger, and it works great with a keyboard.
So anything that requires larger/more screens seems like it might be more productive than a laptop.
One more thing I found strange was the current Xcode 15 beta does not even come with Vision OS SDK (coming later this month) and we cant even play with the WWDC session videos in the simulator
I'm pretty sure iPad's weren't available in advance of the launch day. And, it launched first in the US. I had an iPad app available on launch day and wasn't able to get my hands on an iPad for a month or two. Of course, developing only in the simulator for iPhone/iPad isn't a huge deal...visionOS is probably a lot harder to simulate particularly when it comes to understanding user interaction.
> "I have been a “day one” developer for three of Apple’s platforms"
> "I’m going to be a “day one” developer for the Vision Pro."
> "The Economics of “Day One”"
> "Look for Widgetsmith for visionOS from “day one”."
it looks like a guy who is flexing his "day one" access and using it to advertise some vaporware, and the rest of the words in the blogpost are some fluff maybe gpt could make it
FTX can still be profitable today if they allow SBF to raise funds. They just didn't understand his vision.
Without a doubt, the Vision Pro hardware is a marvel. Having the 3D capture camera in a consumer device would feel like living in the future. The headset design is top notch and seems comfortable. I do have slight worry about how these devices will look like after a year of use.
The holographic eye reconstruction screen. A bit unsettling and strange design idea. We're messing around with people's eyes to make the headset friendlier?
The frosted glass user interface is classy, aesthetically pleasing and very well put together. Lighting and soft shadow interaction with your environment is amazingly well done. I felt the UX the current user experience seems rather lackluster and limited to a 2D format. The first iPhone also had an old fashioned interface, so my guess is they are building the compatibility bridge for the first release. Maybe the next iteration would be actually spatial and not just panels floating in space?
Moreover, the over-reliance on swipe and tap gestures is a point of concern. This design decision is focused on media consumption and limits the overall versatility of the interface. Feels like a very low-bandwidth control method.
A significant drawback, for my use at least, is the lack of control over the hardware. Developers are restricted to building Unity applications within a sandboxed environment, which is a privacy feature intended to prevent apps from accessing the eye gaze point. Good idea, better than what Meta's doing. But too restrictive and frustrating for a developer wanting to run their apps on the bought device.
Apple is once again marketing an expensive device-platform that caters mainly to larger corporations looking to offer their own 'immersive' experiences. Everything about this screams 'coorporate'. I would like to be proven wrong with some creative apps, but those will probably come through the web browser or streamed from Mac, and not as approved native apps running in gimped Unity engine.
Nevertheless, the Vision Pro sets a new benchmark for future headsets. Maybe AR/VR is slowly entering the mainstream?
It may be powerful. It may work very well in some ways (there may be limitations without hand held controllers like an Oculus or PSVR2).
But that’s not what Apple is trying to sell. And they clearly don’t want to be pigeonholed as another fancy gamer device.
Apple has done very well while ignoring the gamer (as opposed to people that play games) customer. I don’t anticipate Apple making big strides towards prioritizing the so called AAA gaming experience now.
Of course, it depends if Apple allows it.
Unless you mean the programming of 3d transformations and such. That is, of course, still hard. Most of the platforms have done a ton of that work for you, though.
speaking of 3d modelling, I taught myself this year in my spare time over a few weeks, and feel im pretty good at it now, ive made scifi wargaming terrain im happy with and almost considering selling some. Blender is free and good and there are literally endless high quality tutorials for almost any specific object you might want to make and after ten or so you will be able to generalise to anything simple you choose. Of course human bodies are more complicated and I was intimidated but I found sculpting caricatures fun despite not being an artist in the slightest after following an excellent tutorial
I do this all the time with the Quest Pro. It's resolution isn't quite there but in most other respects it enables the same type of transportation out of current context to somewhere else. Given you can get these now for 1/4 the cost of a Vision Pro, I think dev interested in Vision Pro should pick up a Quest Pro now and then you have both ecosystems to play with / compare and ultimately, if you are building something where it works, ship it on both.
1. I tried working in my Oculus Quest for about a week (two years ago) and gave up because the resolution is way too poor, and it's too heavy.
2. The price tag.
It looks like the Vision Pro is not a monitor/headset, but rather a computer + monitor all in one. Is that accurate to say?
-
I am tempted by this future, but I don't see it becoming adoptable for any reasonable users for another 5 years minimum. Plus, for developers to truly adopt it, we'd need to be reasonably sure our workflows (e.g. keyboard/mouse sync and ergonomics, portability, etc.) are increased in speed/comfort to ever consider adopting such a feature.
I wonder how we could code in three dimensions or make it feel more physical like wood working somehow.
It’s all text based so I can’t really think of how you’d do it.
HoloLens found it's niche in commercial sectors like manufacturing and logistics so there's a potential path to make a profitable software business in that direction.
However I'm curious what makes this play exciting for indie developers who might not have enough cash for a "long play" here. For iOS it wasn't difficult to see what the benefit would be and iPad only expanded the audience (and breadth of applications having more power under the hood). This is quite a bit different: the technology has been here for a while but the audience seems to be missing (or not as big as you'd think).
What's the play here for small businesses? Just use a portion of your budget to throw spaghetti at the wall?
If Apple wants AR in their normal non-gaming, non-niche segment to work out, they are going to have to come out with a sleek $1000 device in a couple years. They’d have to be very stupid to not realize that, and it couldn’t possibly be the case that you get a trillion dollars by being stupid, right?
They could be signalling that they only want a small number of users for these things.
> it couldn’t possibly be the case that you get a trillion dollars by being stupid, right?
... I mean, the fossil fuel industry likes to think it's smart but what it's doing is incredibly stupid. I'm not sure Apple got to a trillion dollars on merit alone.
Right now it can't fulfil a $5000 product let along a $1000 one.
So they will need a dramatic ramp up across their partners before they even consider a consumer SKU.
This will likely be similar.
When the iPhone came out, not only was it limited it was $600 and no subsidy at a time most people were getting free camera phones.
It took a while (and a price drop) for people to see enough value to pay the price.
If you look at the iPhone, it hasn't gone down in price much if at all.
I think we're at least 5 years away in terms of tech capability to provide the level of experience Apple is looking for at an "SE iPhone" type price point.
But I agree it’s not getting cheap soon. Maybe to $2k for the non-pro version (similar to the announcement Vision Pro) in a few years. Maybe even $1500.
You want to pay $500? You’re going to wait a long time.
I think you could get there pretty quickly by cutting out all the AR stuff and making a VR-only "spatial computing" device, but I dunno if that's something they want to do.
I think at $1999 for a "base" model, they'll sell millions. I just think that the tech they need to improve (optics and batteries) are hard to miniaturize. Chips and sensors they're the best in the world (in terms of consumer electronics). Batteries are just tough, and despite how good the iPhone is as a camera, it still is easily trounced by dedicated digicams.
I mean… that may have been a niche they targeted, whether they found success is debatable. Seems like it was mostly kept afloat by bloated military contracts. And even there it pretty much flopped.
https://www.theregister.com/2022/10/13/us_army_report_micros...
This reinforces the idea that Vision Pro is for a portable, fully immersive work experience. Which dovetails nicely with the post-Covid notion of working in various locations as convenient/affordable. Bring a Vision Pro with you to "the cabin" or hotel or remote working location, to have a more productive experience than just a small laptop. Or college dorm, where you don't have your own dedicated work space.
I might buy it just for the multi monitor setup and to get rid of my extra USB-C video/usb-c portable monitor and my iPad for a 3 monitor setup in smaller spaces.
[1] a Condotel is where you own the entire condo unit. But it gets rented out as a hotel room when you aren’t there and you get the income minus the property management fees
I don't mean "powerpoint/drawing" work. I mean wall of text, code work.
That's because it hasn't been released yet. All the accounts you are reading are strictly controlled and time-limited demos.
Too expensive to buy however.
Will wait for a substantial price reduction on later versions with bugs worked out ... hopefully in the magnitude of 2 / 3rds less before I'd even consider buying it.
Thats' about it.
Despite Gurman's personal speculation, Zeiss already makes prescription VR lenses for under $100:
https://appleinsider.com/articles/23/06/07/vision-pro-prescr...
I would pretty confidently bet on $99 for Apple, with an outside chance of $149.
But now, seing the Vision Pro, they are going to have a hard time to compete.
The funny thing about this statement is that Vision Pro is in fact the smallest rectangular screen of any device he's written code for.
He's hyped up. That's normal. In time he'll understand Vision Pro doesn't provide any better UX for common activities. In fact it's worse in many ways.
Where Vision Pro may shine is tasks where you need to perceive and manipulate complex three-dimensional objects, as they would be in physical space. I see great uses in engineering, design, art. It'd be great to preview interior design, design cars, architecture, create machinery and so on.
It'll also be great for previewing products, so online stores become a lot more viable than they are now, as you get a sense of size and style for an item in Vision Pro.
It may also be great for education, training, simulations.
It has many great uses. But basic apps isn't it. And most people won't care. This thing sucks to wear for more than 20 minutes. It's heavy and uncomfortable. You can't share your experience with others, either. It costs a lot. And you can't multitask with it. I can walk to a place and do something on my phone.
The input model also sucks. To code, for example, you need to hook a bluetooth keyboard and mouse. Looking at symbols one by one to fingertap would be comically slow. At which point, you may as well just get 2-3 screens and work on a normal workstation. For less money.
For the common user it will be amazing for cooking (tells you what to take next and from where and mix in what order, all that fully with arrows on the screen). Or let's say you want to leave your home and it knows that you forgot your keys and it tells you where they are with directions on the screen like the objective marker in a game.
It will know where things are in your home even if you don't pay attention to them it will have object recognition in place and you will be able to say "Hey Siri where did I leave my glasses?" and it will point you to them.
I'd expect a similar wave of people breaking their expensive Vision Pros if they try cooking with one. It's a terrible idea. First, most kitchens are cramped, full of low-hanging cabinets to break your Vision's fragile glass into.
And then, keeping those open vents around vapors full of fat and tasty food bits is a great way to cover the circuits with grease.
We already have a solution for something telling you what to do next, and it's called an iPad with a stand. A phone also does the job and has much lower chance of incidents than a headset, despite yes, you may need to wash your hands from time to time to scroll down. Or... you can simply use assistive features and voice for that. Siri is going to get a lot smarter thanks to LLM, much sooner than Vision Pro will become light and pragmatic for such purposes.
Regarding this "it'll know where things are in your home", let's use basic logic here. It can learn the layout of your rooms and where your immovable furniture is. But no, it can't know where everything that moves is, because this means you literally can't move anything unless you have the headset on to track its location. Or slap expensive AirTags on every single jar and utensil maybe. All solutions would be hilariously impractical. And... we'll end up with where I started: a broken Vision Pro glass as you slam it in a cupboard while trying to fish out a jar of condiments.
I don't know what is about VR that makes people pull out the fantasy scenarios. It's simply a (bulky) screen with pass through. It's not a wizard. It can't know things unless there's a way for it to find them.
We can imagine a super-thin model that you can keep on your face 24/7 and sleep with it too, so it tracks your entire life forever and knows you better than you know yourself. And it synchronizes with your spouse and children who also wear their own headsets 24/7. And it's unbreakable. And the battery never runs out. We can imagine many things. They don't exist, and won't exist any time soon. "Not on the horizon" as Steve Jobs used to say.
Ambient computing of the style you describe I do think is a common use case and I look forward to less invasive form factors to tackle it.
I have a hunch that coding as we know it is going to look very different in ~5-10 years.
You see, the rules of formal languages that encode formal rules of system constraints pre-date computers by centuries. Think of math proofs, for example. Sure, we can encode symbols as emojis, or geometric figures or whatever. But in the end, it's sequences of symbols, that's the nature of it. And tapping symbols one by one with a headset will suck, no matter how programming looks.
The rules of formal languages that encode formal rules of system constraints pre-date in fact our species too. Think about what DNA is. Oh yeah, spooky, isn't it. A sequence of symbols (GTCA) encoding a sequence of more complex symbols (proteins). Spooky! But yes, DNA is our code. And it works the same as our programming code.
Now I know where you're going. LLMs. Let's assume an LLM writes the code for you. You still have to read it, which you can do fine with a headset (if it's not as encapsulating and heavy, and with short battery life as Vision Pro v1). But if you spot something's off, you need to adjust it. Go directly for the kill, and make that surgical series of edits. You know? Or... maybe you can spend the rest of the day hopelessly trying to explain to Siri 2030 year edition what you want to do, instead of going in and doing it, for that "last mile".
Because if AI can do the last mile itself, to the point you don't need to even verify it... first, that's the fast way to AI shipping code we don't understand and basically giving up our entire civilization to it. And second... we don't need to code, but we also won't need to exist, and therefore not need headsets.
So in the worldlines where we DO exist... Vision Pro sucks for coding, because it's a shitty human interface to editing code.
And in the worldlines where we DO NOT exist... Vision Pro sucks for coding, because AI doesn't need headsets.
You assume you will hate it, maybe you will. But maybe the future won’t involve dedicated furniture to put things on and cables connecting them. Maybe you’ll be able to work wherever you want and with the same amount of productivity. Or maybe even better productivity!
But you’re probably right, the future will never get any better, this device is pointless and will never lead to better versions of itself or point to other ways of working. Thank you, there’s no telling where we might end up without true believers of the status quo like yourself.
LLMs are still a really immature technology. The hype is about where it could go in future, not necessarily where it is now.
Think about when compilers were immature technology, and the science of parsing/etc and optimizations were not well understood. You could make the exact same argument you have made now about the need of editing assembly or machine code by hand when the compiler doesn't get it right.
It was indeed common practice to do this well into the 1980s. That, and inline assembly is increasingly unnecessary now.
Grognards with sufficient disposable income - rejoice! At last you can (for example) play Operation Barbarossa at regiment level - and retain your sanity!