Developer tools to create spatial experiences for Apple Vision Pro
apple.com
apple.com
https://www.economist.com/business/2023/06/06/apples-vision-...
I think it’s totally reasonable, to expect developers to find the real use
"Better Than Life: Las Vegas"
Games, gambling, shows, and porn all in one "so immersive you won't want to leave" experience.
"Disney ain't got nuthin' on this!"
The Vision Pro was showcased with Disney as a content partner. While Disney has historically featured "family friendly" entertainment, there is definitely a market for adult entertainment that the Vision Pro could excel in.
Is it "an exciting Las Vegas adventure in the privacy of your home" or "a lonely person in a dark room trying to escape the sadness of reality"?
FWIW, my take is that this is yet another "slippery" media [0] and will mutate to whatever form brings its creators the most succe$$
Apple won't allow alternate stores on Apple-branded mobile devices, or sideloading.
This is a fundamental disrespect of their customers who have purchased the device. Apple regards their brand image more than allowing their customers to display what they want on the device they own.
Combine that and pornography and I think you’ve nailed it. Pun possibly intended.
- Janet, the Good Place
And again with Objective-C it's impossible to prevent private API usage unless you have some sort of App Store model which can inspect the binaries and prevent abuse.
So it's ultimately going to be the same as iOS.
A alternative is to sandbox applications to prevent them from calling anything else than the official API, and to use a less restrictive sandbox for applications signed by a key owned by the vendor.
Yes, yes they could. That's not and shouldn't be Apple's problem. That's your workplace's problem to regulate how the device is used on-site, the government's problem to regulate how it can be used in public, your household's problem on how it can be used in private, etc.
The device will be inevitably jail-broken anyways, so a walled-garden isn't going to stop bad actors.
Not to mention, most of the things you mentioned can already be accomplished with less expensive and much more subtle devices, like a standard digital camera. And those device definitely don't try to prevent abuse. (Imagine if your camera refused to take a picture because it thought you didn't have permission!)
Apple can't just capture them and then allow any rogue app to access it. The device would be considered a threat to national security and banned.
And no there are no other mass consumer devices which specifically store a 3D representation of your face and a high resolution scan of your retinal pattern.
Considering the vast majority of exploits on Windows are not the fault of Windows and are the fault of 3rd party applications. The fault is always put on Microsoft.
If Apple gives you free rein, and shit hits the fan. People won’t blame the company for allowing a piece of software to go rogue. People will blame Apple.
As far as I’m aware you can’t do that with iOS devices (potentially including the AR ones).
Sure you can. You can fire up Xcode right now and build a self signed app to your phone with whatever the heck you want in it. Distributing that on the App Store is another story, but there's nothing stopping you from executing whatever arbitrary code you'd like on your own phone.
Try altering the iOS home screen behaviour, directly accessing the filesystem, writing your own device driver or tweaking the kernel. You certainly don't have "bare metal access" except by voiding your warranty and risking bricking your device by jailbreaking it, if there even is a jailbreak exploit that currently works on your device.
The first computers were operated by the British and US defence establishments. Could you explain in which way you feel ENIAC was "100% open platform"?
The word is meaningless as you use it.
After the Nvidia blamed them for their failing GPUs, they literally never shipped anything Nvidia ever again. There’s probably an alternate universe where metal does not exist because Apple and Nvidia became friends and just optimized cuda for macOS
The most classic game theory example ever, the Prisoner's Dilemma, is solved by cooperation when played iteratively instead of one-shot
In general, lots of systems can reach an equilibrium of cooperation
I remember Jerry Pournelle on Twit.TV saying he reviewed the original Mac and said it was a bold vision but ultimately a toy. A great idea but with not enough RAM to make it really useful. Steve jobs saw the review and took action. Jerry said the 512KB Mac was excellent but that point it was too late.
He was black listed for life from having any early access to hardware for review and was never allowed to an Apple conference ever again. This was in place until Jerry's death 33 years later.
Don’t expect long term love for UE from Amazon or Sony or Microsoft, either. They all know that Epic’s end game is to get regulators to force platforms to allow Epic as a competing app store. Let others take the risk, swoop in and undercut once there are profits to be had.
Nobody is going to want to fund that. It will take a decade or more, but UE is doomed unless Epic divests it or gives up on the predatory app store / regulatory business model.
Take the risk? What risk? By now both surviving mobile platforms (iOS and Android) are well established, and especially the iOS app store is raking in obscene profits for Apple out of the work of others. I would have been ok with a 30% "app store tax" a few years ago, but nowadays it just feels like milking the cash cow...
I’m just saying that Epic is using a legal strategy to attack markets with vertically integrated hardware and software distribution (phones, consoles). The owners of these platforms are not thrilled, and we should expect UE to be a casualty.
I’m surprised that’s emotionally charged for you.
- godot - bevy - flutter - qt
since all those above render their own ui widgets - you need to integrate somehow this eye tracking to be able to select it.
Politics aside, nobody should ever choose Unreal Engine for standalone VR anyways. It's clear that Epic doesn't prioritize VR, so it's more easily explained as an efficient use of resources.
In concrete terms, Unreal Engine's VR support is awful; you can't get a good looking game running at high framerates on mobile hardware with it. Its forward renderer is severely lacking on mobile VR, and deferred is untenable.
I don't want Unity to be as dominant as it is, but it's the only sane choice for VR. Epic is very clearly not interested in being a leader in this area.
> from my point of view “figure out what it’s for” is the ultimate computer hackers playground
I don't have much to add to this fantastic comment.
To calculate the bridge spans requirements it took 3 guys a 3-5 days to do the calculations, drafting the spans, and double checking everything. My father in law opened the book on basic and saw the math it could do.
He wrote a very basic program to take the inputs and do the math. It took the computer less than an hour to return the results.
Who knows that the future brings but I’ll be optimistic.
Touch found it's place as the interface for small-form devices. I'm pretty sure VR is the interface for gaming (with other niche applications). It will be wildly popular, probably a headset in every house within 20 years, I just suspect it will be doing what we already expect - gaming, visualizing chemical reactions, architecture, etc... things we already do in 2d, made just a bit more convenient in 3d with spatial controllers.
My supervisor, the head of R&D, had an SGI w/ 3d goggles and a glove and would spend hours in front of it manipulating virtual molecules. He would talk about his days as a PhD student at Stanford in the early 70s working with punch cards and the progress of computing up until that time just over 20 years later… and thought that maybe in 10 years, when his rig that cost about $100k would be cheap enough for the average consumer, it would be the next step in personal computing.
Following developments in VR/AR for a while, I've felt like we've been in a "business computers before spreadsheets", "home computers before the internet", "mp3 players before the iPod", "PDAs before the iPhone" sort of period, wondering if we'd see a sea change platform emerge. I generally have a low tolerance for hype, but I'm excited about the potential here. Can't wait to experiment with developing software for it!
You sitting on your couch as virtual events and shows happen around you.
My second computer I got for a graduation gift was a Mac LCII with 10MB RAM with a 512x384 12 inch monitor a LaserWriter LS printer, an Apple //e daughter card, a 5-1/4 inch drive for the card and a copy of SoftPC. It was $4000 - in 1992.
People use to spend a lot more on computers back in the day.
Both of my computers were hackers playgrounds for me.
I would have gladly paid $99 a year for the bragging rights of putting my app on the iOS App Store so anyone could get it.
I paid for a 65C02 compiler back in mid 80s using my allowance and paid for a Mac C compiler in the early 90s.
My parents were solidly middle class - a factory worker and a teacher.
If this runs a version of iOS, we can't access the code that is running in it, we can't open a terminal, we can't use our programing language of choice. We can't use the actual device for building anything hacker related by itself.
We can make apps for it using an actual PC made for building, but only if it runs MacOS, ans using their proprietary language. The actual device is far from a hackers dream or plaything if its as locked up as the iPhone. But very cool device though, hopefully it brings VR into Mainstram outside of gaming
iPhone enthusiasts Stephen Hackett (his real name), hack into pre-release versions of iOS all of the time to see how it’s running from their computer.
Even that 1986 6th grade hacker that I was would never want to program on the phone. I fail to see the fascination with running a fully fledged IDE on a phone.
When I was programming on the very open Windows CE devices pre-iPhone, I avoided running on the phone as long as possible and used the emulator.
You can use any programming language you want that compiles down to ARM.
This "you can build and directly deploy an app to a limited number of devices if you're on the developer program and have a mac, therefore you can 'hack iOS all you want'" meme is borderline dishonest.
Also, the fact that you "fail to see the fascination" is of little significance to those who do want to develop on-device. You don't get to make "you can hack iOS all you want" work by restricting what people are allowed to want.
Messing around with the kernel generally does run the risk of voiding warranties and bricking devices.
And most of the things I'd want to do with an iOS device that I'm prevented from doing wouldn't involve any kernel-level work anyway.
But more importantly that's the most pathetic justification I've ever heard. By that logic, since playing the piano is difficult we might as well sell pianos with all the keys glued together.
The truth of it is that Apple products are not designed for tinkering and they actively resist attempts to do it. To spend my time plumbing their internals to add value to a product that treats me an my kind as a sort of infection would be a pure waste. I'll do my hacking on Linux and use my iPhone for arguing with people on hackernews, it's fine.
nut up. hacking has never been perfectly safe and has never had a promise of easy. if you want it enough, you can make it happen.
There are plenty of kids out there tinkering and hacking in the truest sense on iOS and macOS using one of their relative's dev accounts.
The point is that you can hack the device to run your software, or distribute your software so that it's easy for others to use it as well, but not both.
Problem is, Apple isn't advertising this as a phone or "mobile device". They're advertising it as a spatial computer. And IDEs are probably my second-most-used computer applications after web browsers, so I want to be able to use them with the near-endless monitor space offered by this spatial computer.
Do you also want to run an IDE on the Apple Watch?
I get what you are trying to say, but this just isn’t true.
The cost of deploying to App Store is negligible compared to the rest of the HW needed
Some playgrounds have no fences, but also no interesting objects to play with.
Other playgrounds have boundaries far out, and comes with a huge combinatorial explosion of possibility.
The price doesn’t have to come down as 3.5k is the modern day equivalent to the price of a laptop 10-20 years ago.
People wallet and their salary is not out of rubber - there is limit how much they can stretch their budget.
There will be multiple VisionPro competitors within half a decade and they’ll all compete on price and focus on mass adoption - just as it happens with phones and laptops.
Bill Gates once said that in some ways Microsoft was lucky to be as succesful as they where, that they had compititors that had better products but to Microsofts luck, could never get a foothold in the market.
Maybe Apple Vision is the future but it could also just end up as another path not taken on the trash heap of history. Time will tell.
Survivor bias is real, but the interesting failures (in comparison to VR) are things like 3D TV.
3D TV has a very weak effect, the 3D effect is basically dismissed after a while, after you get immersed in the actual content. A bit like watching a (good) silent movie, at first it's jarring, but after an hour you don't notice it anymore.
On the other hand, because VR is so immersive, the effects on the brain are quantifiable, it is fundamentally different than all other forms of media. You can permanently change your brain with VR in ways that are literally impossible with letterbox media. It can change your brain, in a way that no technology has ever been able to before ('Strategic modification of Bayesian priors' if you're into the science).
That kind of thing is why VR is a guaranteed success, it's not just about the fact that movies and entertainment and games will get to a new level, but the psychological possibilities for actually modifying the self with VR are going to ensure we all have headsets in a decade.
VR is immersive to a degree that it tricks your brain completely. It can activate powerful biological mechanisms that cannot be done in other digital/scaleable ways. For example, there's a company in Spain called Virtual Bodyworks, founded by the VR researcher Mel Slater, which can affect your implicit bias with body swaps performed in VR. This is genuine selective neural net-surgery, done in a way that cannot be replicated outside of VR.
The company I work at has a smoking-cessation VR app, takes 5 minutes for some people to go from decades of unbroken addiction/40 smokes a day/first smoke within 5 minutes of waking, to not even being able to think about smoking. There is no way to do what we do with a phone app or even a 3DTV, it requires VR.
That kind of software is going to be the reason VR will take off, it's capable of changing the mind in ways we can't imagine at the moment.
Not to mention that spatial computing is the 'final form' of human machine interaction, plenty of research on that.
Five minutes to quit smoking? There is one app that claims to be clinicly proven and even they only claim 33% of users actually quit after completing the program.
Most of the tobacco cessation products top out at 33% for some reason, but combining them gives higher success rates.
What app are you referring to btw? Very interested in this area at the moment.
> we [will] all have headsets in a decade
you do mention a couple things I hadn't considered or heard of before.
> The company I work at has a smoking-cessation VR app
This sounds very intriguing! Could you possibly provide some references and/or the name of your company? (I don't smoke but I'd love to understand better how this works.)
More generally, how do you stay on top of what's happening in VR tech & applications? I'm asking because I've clearly been missing out on some significant developments.
>> we [will] all have headsets in a decade
I'll take that bet. I'm a kind of tech-cynic, have a Nokia 8110, ten year old thinkpad and listen to minidiscs. But in January I'll fly to the states and buy a Vision pro day 1. The last time I was this excited about a technological innovation was with OS X. My brother was a CS guy and he recommended I buy an iBook when I started university because he thought OX was going to be a game changer. And in the beginning it was weird to have the only Apple in a lecture hall with 300 Dells. But 10.1, 10.2, 10.3... It felt like being on the cutting edge.
That's what the Vision Pro looks like to me. The Powerbook 12" all over again, future-tech.
>Could you possibly provide some references and/or the name of your company? (I don't smoke but I'd love to understand better how this works.)
It's really simple as hell, there's an effect called the 'Garcia' effect, whereby if you feel sick ≈6 hours after eating or drinking a novel taste, you get a lifelong aversion to that taste of smell. It's highly conserved, found in every animal studied with the sole exception of vampire bats (they only have one dietary option).
And this was used in tobacco cessation in the 60's. There's a lovely case study where a 14 year old boy is given apomorphine three times while smoking, he vomits so copiously he passes out at one point, but literally can't smoke afterwards.
It fell out of favour because injecting people with opiates to induce uncontrollable bouts of vomiting wasn't a popular way to treat smoking.
We just do with a simple VR spinning room, you take as long a break from smoking as you can comfortably manage, light up and then do our 'aversion' session for 5 minutes, or as long as you can manage.
From that, we see around 20% of people that are unaffected (the efficacy is dependent on a factor that varies ≈10000:1 across a normal population).
For most people, you get some effect, and for around a third you get an instant reaction where even 2-3 5 minute sessions mean they just can't smoke. They can cave to cravings and light up... and then just have to throw the cigarette away in disgust.
Nicorette meets Clockwork Orange in a way.
>More generally, how do you stay on top of what's happening in VR tech & applications? I'm asking because I've clearly been missing out on some significant developments.
Very difficult, so much happening in various silos. Jaan Aru and Mel Slater are probably the two researchers I try to follow as much as possible, but this is a prescience at the moment, so even finding common terminology is a challenge.
Of that 50%, around a half or so have a history of this with some kind of alcohol. I have one friend who will almost puke if he smells whiskey, because of one terrible night in Norrköping with a bottle of Paddy's.
Will it also cause aversion to headsets and to VR? :)
Unlikely, the Garcia effect is tightly coupled to taste/smell. Could inadvertently give a lifetime aversion to any food/drink though, especially if it's a new taste.
Also, I tried looking into the Strategic modification.. paper? but didn’t find it.
And yes, it's creepy for sure. There is a theory that every generation witnesses a key technological development and is 'lost' to that development, and the generation that grows up with that technology is immune to it. I imagine VR is going to be a technology like that, there will be people 'lost to it' in the same way that people were lost to TV or to smartphones today ( ref: https://www.youtube.com/watch?v=6Olt-ZtV_CE ). Ethically VR is going to be a minefield.
Here's the paper that coined the phrase:
https://www.frontiersin.org/articles/10.3389/fpsyg.2019.0035...
And here's I think one of the better papers giving that explanation in terms of VR.
https://pubmed.ncbi.nlm.nih.gov/30553934/
Although I come more from a mechanistic/biological background, but there the research has not come so far, AFAIK.
Although Catherine Dulac and Christian Broberger are on the cutting edge in that area.
https://www.youtube.com/watch?v=xtiZb-kuexU
That lecture is a fantastic place to start, and the statement at 6:55 is key to seeing why VR is going to be a game changer.
*edit*
Maybe also this lecture too, the shear scale of the visual system in the brain is another key aspect to how VR can be so powerful.
I find there is a similar acclimitisation effect to 3DTVs. Do a repetitive task in VR and it doesn't take long for VR to not matter. If you use VR everyday the wow disappears and desktop gaming can be a refreshing change. Only after a decent refractory period might you put on a headset and get some sense of magic again.
> takes 5 minutes ... no way to do what we do with a phone app or even a 3DTV, it requires VR.
Oh, you are snakeoil salesman. I regret responding.
Emphasis mine.
This is a meaningless statement.
I argue the only valid statistic is x% remain non-smokers at y years, something comparable to other methods.
And even if your company’s method does turn out to be statistically significant, that doesn’t necessitate everyone owning the device, only the clinic needs to.
I thought about this a while back, you could definitely do aversion therapy treatments potentially more effectively with a VR setup.
> that doesn’t necessitate everyone owning the device, only the clinic needs to.
If this approach works, then it will be super widely adopted as it costs insane amounts of money to drive behaviour changes right now, and lots of health services would be interested.
Absolutely agree, but our problem is time, we did our first patient around 8 months ago, so we're just limited by that timeline.
So far so good though, followup shows classic Aversion reactions, the people that were affected still have a very strong reaction to the taste and smell.
A bit off topic but in this context I found “Dogville” to be interesting.
Also makes me think of 'The testament of Dr. Mabuse', totally incredible film.
I've played with VR, and it's really not that immersive - no more than any other good media.
In particular, VR is definitely not as immersive as actual reality, so anything that could be done in VR to convince someone of non-fantastical things can also be done (at a higher expense, for sure) with actors and props. So, your claim about 5 minutes to quit smoking has no reasonable chance of being true from where I'm sitting.
Well yeah, it can be done in real life, it's just much more brutal.
https://www.sciencedirect.com/science/article/abs/pii/000579...
https://sci-hub.st/10.1016/0005-7967(63)90042-1
A fourteen-year-old boy was said by his parents to have started smoking at the age of seven, and to be spending every penny of his pocket money on cigarettes. He had at one time regularly smoked 40 cigarettes per day, but was now averaging about half that number because his pocket money had been reduced. He said he wanted to give up smoking because he had a smoker’s cough, was breathless on exertion, and because it was costing so much money. Physical examination and chest X-ray were normal.
Treatment was given in the outpatient department. On the first occasion he was given an injection of apomorphine l/20 g, and after seven minutes he was told to start smoking. At eleven minutes he became nauseated and vomited copiously. Four days later he came for the second treatment, and said that he still had the craving for cigarettes, but had not in fact smoked since the previous session because he felt nauseated when he tried to light one. He was given an injection of apomorphine l/20g, and after seven minutes he lit a cigarette reluctantly, and immediately said he felt ill. He was encouraged to continue smoking, and he collapsed. He was given oxygen and an injection of Coramine. When he recovered he was very hungry and asked for food, which he ate voraciously. Four days later he was given apomorphine l/40 g. and vomited as soon as he attempted to light a cigarette seven minutes later.
When he next attended he said he no longer had any craving for cigarettes, and he made two interesting comments: “When f see an advert on T.V. for cigarettes, it seems like a dead advert. ” “Just smoke from my father’s cigarette makes me feel ill”
Two months later he left school and started working. He said he had “got a bit down” at work and wanted to “keep in with the others”, so he had accepted a proffered cigarette. He immediately felt faint and hot, and was unable to smoke. It is now a year since his treatment, and his parents confirm that he no longer smokes.
hmm, this seems to be a rather biased description of the history.
A big part of it was also Gates ruthless business practices.
Bill Gates famously engineered his own luck by opposing a free market.
I am not sure why you are focusing on "survivor bias". My comment was about being a programmer and the fun of seeing a new technology that is an undefined territory to explore.
But I did directly mention that many of these do not workout as as I referenced: amiga, os/2, palm, and VisiCalc and Lotus 123
However, so that you may have some idea of my personal "survivor bias", here is a super simplified multi-dimension history of my over 43 years of computing smashed into a linear list with massive things forgotten:
TRS-80 Model III, LDOS, BASIC, VisiCalc,
modem-to-modem
TRS-80 Model 100
CompuServe
MS-DOS compatible, MultiPlan, TurboPascal, dBase
Windows 3.0
DesqView
Delphi Online
OS/2, REXX
Macintosh System 7 --> 1993 to OS X
Dial-up ISP
Solaris, Perl
BBEdit
Windows 95
Palm Pilot
Linux for Servers --> 1998 to Today
Linux Desktop
ColdFusion
Apache
Python --> 1999 to Today
Always on Internet (ISDN, DSL, Cable) --> 2004 to Today
Nokia cellphones
PHP
Zope/Plone
Mac OS X --> BETA to Today
Windows XP
Nginx
iPhone --> 2009 to Today
SublimeText
Windows 10 --> 2018 to Today
VS Code --> 2018 to Today
NodeJS --> 2018 to Today
SvelteKit --> 2021 to Today
KDE Neon --> 2022 to Today (the year of Desktop Linux arrived for me)
As far as Apple's Vision Pro, I have no clue if it will be successful. For myself personally, it is the first headset I am even interested in playing with.
I have had one or more Macs since 1993, I went years without a Windows system. (Servers have been consistently Linux with some Unix and Macs early on).
I probably spent less than a day total with Windows for Work Groups, NT, 98,ME,Vista,7,8,11, etc
Much of my Windows 3 usage was under OS/2 Warp
The above list really about my personal major themes. I already can see major omissions, but Windows is not one.
I saw Personal Computer, PDA / SmartPhone and Tablet and I never doubted their success as a product category.
But Vision Pro, or AR / MR is the first one I am joining that camp. I am sure It will do well in many areas, I just dont see it carry the same weight as PC, or Smartphone.
I worry though: is the passthrough reliable enough to use a sharp kitchen knife safely while wearing it?
I think the Apple demo of people mostly sitting on a couch is not a lack of imagination, but a pragmatic approach of how this device should be used.
My intuition regarding proprioception is similar to yours, though -- give me laggy input of my own hands and I'm still pretty likely to get things right.
There is a huge difference between consistent latency and jitter, but even then I doubt that 1ms latency + 1ms jitter would be very noticeable.
I’m definitely getting a vision pro the moment I can, but I am most definitely not going to wear one when I’m handling anything other than a keyboard on my couch.
I'm planning to be a Day 1 user of AVP to see what I can build with it, and I look forward to your cooking app!
Imagine just the eye and hand tracking with no screens. That could be a much smaller device and one can trade the screens for an interactive environment.
https://www.youtube.com/watch?v=CavlglPYVvQ
edit:
They already seem to have a beta VR application: https://aerodrums.com/3d-updates
Thank you for the plug. We're doing a Kickstarter right now for Aerodrums 2.
It will have support for VR passthrough. We'll add support for Apple Vision Pro and Meta Quest 3 as soon as they make it possible (if anyone has connections to speed that up, I am interested).
The project page is here: https://www.kickstarter.com/projects/aerodrums/aerodrums-2-a...
I make Aerodrums and that operates at 125Hz for the mocap. As far as I understand, Apple Vision Pro tracks hands at 83Hz. They would need to improve that (and provide developers what they need to do a good job determining the position of the drumstick tip), for an air drumming app to be possible without accessories that plays to a musical instrument standard.
I've replied elsehwere in the thread plugging our live Kickstarter for Aerodrums 2, check it out, it only has 7 days to go.
The idea of a portable large viewing screen is brilliant but that it not the real main selling point of this. A decent VR head set could do something similar.
In 10 years time, comments like this will either look like a prophet that could see the folly of the future... or be like all those people that dumped on the iPod and iPhone.
Are there any VR sets & OSs that can project multiple app windows into the space around you?
It is conceivable that completely different categories of workflows will become available. In the end everything we do on a computer has been invented at some point or another. I guess there are plenty of eager early-adopters and fanboys that will be more than happy to report on their magical productivity boosts.
In any case, assuming that the above caveat is not an issue, if the main real use case is as external monitor replacement I would think that suitably optimized hardware would deliver much better value for money. No need to wear a computer on your head when you need to carry one on your laptop (even mobile) anyway.
I’m also not really concerned about value for money. As far as business expenses go it wouldn’t exactly be a major spend. I’m personally more concerned about how good the experience is, and value for weight and size. For about half the days in the year I have to live with ~35 lbs of possessions that have to be able to fit in a suitcase. So for me if it could deliver sufficiently improved work experience it would be worth it.
Meta completely failed in the strategy, their store is empty of AAA offerings. This is not bad luck it is a blatant incompetent strategy from Meta. It is in the business book.
I assume Apple has a strategy, it could fail? Obviously, but they are doing what is right.
When the iPhone 1 launched there were many hiccups but in a short period of time.
I agree that initially this product will not sell many units. A sci-fi thinking also tells me that this could improve many industries with completely new interactions with the world. I see this more a B2B product than B2C.
- SDK on Xcode 15 beta 2: https://developer.apple.com/download/all/?q=xcode%2015
- Developer documentation - visionOS: https://developer.apple.com/documentation/visionos/
- Plan your visionOS app: https://developer.apple.com/visionos/planning/
- Human Interface Guideline - Designing for visionOS: https://developer.apple.com/design/human-interface-guideline...
- Apple Design Resources - visionOS: https://www.figma.com/community/file/1253443272911187215/
- App icon tool (Parallax Previewer): https://itunespartner.apple.com/assets/downloads/Parallax%20...
- visionOS Beta Release Notes (new features, known issues): https://developer.apple.com/documentation/visionos-release-n...
You still need to pay for more advanced features, and to publish your app.
I think it will work a bit like wireless headphones with audio passthrough. I can sit at my PC and hear audio from whatever I'm doing, but I can also get up and move around and still hear that audio. The audio passthrough, even though it's not perfect, is easily good enough for me to hear my environment and even carry on a conversation without having to take them off, and I can pause whatever music or podcast or video I'm listening to just by pressing a button on the side.
I think the headset will provide a similar experience in that you can push whatever random YouTube video you were watching out of your direct line of sight, and then get up and go make yourself lunch while you're still half paying attention to it. Not only that, but you can interact with it using your eyes + minimal hand gestures.
The really intense, fully-present experiences (like gaming on a current headset) will probably still happen, but I think most people won't be spending much of their time there, similar to how the total amount of time spent in modern, intense, realistic gaming is tiny compared to how much time people spend scrolling feeds and mindlessly consuming content.
Or have a similar experience to the living with lag video, this time no because time lag, but eye hand not coordinating.
My experiments using straws in VR just led me to stabbing myself in the gum, lip, nose and cheek in various painful ways. The perfect vessel is a long neck beer bottle. Easy to get into your face-hole and won't bang against your goggles. The downside is how easy it is to knock over.
No, but they will mess with and override proprioception. I've used the Vision Pro and the first time you try to take a drink with it you'll probably miss your mouth. I think the main reason is that you see the cup coming towards your slightly offset "eyes" and your body automatically adjusts to match.
I think if I had closed my eyes (I didn't try that) I would have done better.
Probably would want to practice with water, not hot drinks.
My bet would be there is none in our generation and we spend the rest of our life with two kind of computing: one that fits our hand and other devices to deal with the rest.
The question becomes, is this inherently better and more convenient than a laptop ? If it's not, it will stay a nice accesory, like the watch is a nice accessory.
- That can be used to zoom in (binoculars)
- Keep track and show the score of the real life fencing, ping pong, tennis, card game ,etc (Apple Score) your playing
- See how a building or place looked x amount of years ago (Apple Rewind)
- Go to a conference and upon meeting someone the glasses provide their name and company their with
- Many many more innovations
and LOL my nicknames for some of those ideas are probably silly, but they are unique and i think useful to the point of people buying Apple Smart Glasses as much as they buy iPhones.
All of your examples are great, but you'd still keep a phone in your pocket while doing these activities.
Another aspect to this: most of those were better done with Google Glass, but the social discussion and Google being Google just killed it. I hope the Apple devices will bring more acceptance, but once it's there I'd see other AR makers take precedence to bring better implementations than what the Vision Pro is pointing at.
Heck, at this point the iPhone isn't even the global market leader, I'd totally see a Samsung like maker take the AR market and push it further, the same way their smartphones are actually pushing the enveloppe while iPhones and Pixels keep being the vanilla choice.
Now The iPhone could do the ideas ive thought of above but it's all about User Experience ... when playing ping pong im not going to hold my phone up and let it take score for me and display it on my phone. No i want this to be done automagically and effortlessly and wearing glasses is the best UX for it so is meeting people and seeing their name/info and all my other ideas I noted above and others not noted. One of my coolest ideas ive thought of fixes an old age question we as humans wont ask again.
Meta (Facebook) not having a phone on the market to sell and work alongside smart glasses is a bad thing for them!
The watch itself is already a phone substitute for many people and continuing to grow. Isn't wearing a small thing on the wrist better than a brick in your pocket?
Do you have any imagination about how form favors could advance or even be different?
The phone is not the final destination for personal computing.
Again...
I’d love to read a comparison between visionOS SDK and OpenXR from someone who has actually used the latter. I gather that OpenXR is lower level, but what exactly does that mean for an app developer? How much more work would it be to build a “spatial experience” cross-platform with OpenXR? What would the compromises be for both approaches? Would it be feasible to build an engine that runs on both platforms?
Mostly that they won't be using OpenXR directly. It's a standardised layer that lets many kinds of tools and libraries make use of the same hardware.
That’s a question that I find interesting. Even if I don’t end up building such an app, it would illuminate the differences between Apple’s SDK and the rest of the industry.
In the WWDC unity-metal demonstration they definitely seemed to have gaze tracking. Have you checked looked through the sdk and confirmed that there is no public way to do this?
Even the Unity team had to correct a few edge cases around WWDC as the sessions dropped:
https://twitter.com/pvncher/status/1666476686576332800?t=9Br...
---
I'm specifically talking about the "Bring your Unity VR app to a fully immersive space" WWDDC talk, which claimed to be built on Metal and ArKit, and had a very very brief demo in it talking about gaze, screenshot here: https://media.hachyderm.io/media_attachments/files/110/520/9...
Note the demo wasn't actually on-device, and gaze was just following where the screen was pointing, so it's definitely possible I'm reading too much into this.
See this : https://twitter.com/Tojiro/status/1667028702734209025?t=1SnM...
EDIT: Checked with our team and so VR mode gaze has event triggers on any Apple UI object without exposing the details so that works unless your UI is not easily portable. And that's the same across all MR and VR modes.
Clearly in the comments it just shows how out of touch they are in reality. There will be a version of the vision product line-up that will be reduced to the size and naturalism of a pair of glasses with some AR and reduced XR capabilities. But that is 5 - 10 years down the line or possibly could be quicker than expected.
Who knows, but this first version isn't it. This one is for the devs and techies only.
https://9to5mac.com/2023/02/23/apple-headset-cheaper-second-...
I've read a hundred general comments about Apple's strategy and market fit. Can we just talk about the actual topic of this post for once?
Doing CAD work in AR with a $3500 devices is worth it for any businesses.
Even as a sale tools when selling large products (cars, boats, etc.) without the need for stock. i.e. I can go to my local dealership, and look at their new model in persons. Then play with AR to decide which options I want for the colour, wheels, dashboard, etc.
I downloaded the SDK and gave it a try. It was pretty seamless to get the sim booted up. However I did have trouble getting any of my hobby apps to build for the native SDK. A lot of libraries will need to add target conditionals to build natively. I finally got one of my most simple apps running, and SwiftUI does translate pretty well into it, but there’s still a bit of visual jank that I guess I could probably learn how to fix. The simulator also does seem kind of slow and unresponsive; I wonder if that’s at all representative of the final OS.
https://www.polygon.com/23613576/jeff-minter-profile-akka-ar...
The experience would be wild but he has a habit of also releasing a lot of stuff on doomed systems. Atari Jaguar, PLaystation Vita, Nuon for example. That said when you have released like 100 games, eventually you will launch on platforms with no future.
But having Tempest come out of the walls... oh man!
EDIT: It was an app called Wavepaths, here is a link to the Rolling Stone article. (https://www.rollingstone.com/culture/culture-news/hot-digita...)
EDIT pt.2: Here is the app itself. (https://wavepaths.com/)
It is academic work though, so the salary is not very competitive.
But yeah, it is low. In particular they seem to be looking for an ML researcher who is supposed to get even less.
https://en.wikipedia.org/wiki/List_of_European_Union_member_...
We ended up realising we were way more fascinated by the signs than any of the animials.
Then ended up hysterically laughing when we realised that we were at a zoo mesmerised by everything but the things we planned to be mesmerised by
This was the inspiration for his part story album "The Point".
On the negative side, I think it the Vision Pro just needs a HYPER DEMON port.
The reviews are a treat (seizure warning):
https://qri.org/blog/psycrypto-contest
The idea was to make visual content were the message can only be received while tripping.
I do ceramics on the wheel, and having an app that can show me techniques, highlight areas that are thin or weak, critique my technique in real time, and so on, would be something I would definitely pay for.
Same with painting.
I can't wait to see the AR apps!
Although it might not matter - even cracks on a camera lens affect the picture less than you'd think, because they're out of focus.
As an alternative - would a headset lite version without the AR work better here? Display needed information on a nearby tablet or large tv while the user wears a headband only.
The windshield has a stack of huge stickers on it that get torn away by the pit crew during stops. Apparently you can get stacks up to about 20 layers.
Seems like a reasonable possibility for messy AR applications.
I'm keen on trying it or getting one, but I have zero interest in using it while cooking or similar.
Could even put dinosaurs in there.
And with a browser in place, I can watch painting tutorials while actually painting, and not worry that I dont own a specific brush or other tool.
It's awesome. It's on Steam for anything that supports SteamVR or PCVR, and on the Quest store for Meta devices.
(I probably would recommend waiting for Quest 3 in a couple of months though ... it'll be so much more like the Vision Pro ...)
The Quest 3 may be better but literally everything they asked for requires good passthrough.
That 1/7th the price would be sunk money if they’re trying to evaluate real world interaction.
https://youtu.be/7KCez6Ocxxg?t=40
So did Marc Gurman:
https://www.bloomberg.com/news/newsletters/2023-05-28/meta-q...
> video pass-through on the Quest 3 presented colors more accurately and offered an almost lifelike rendering of the real world. I was even able to use my phone while wearing the headset, something that often feels impossible on a Quest 2
It will certainly be fascinating to see how good it really is.
honestly, for it's primary purpose it's great. That is, you get complete situational awareness, you can see people and talk to them, pick up your coffee and drink from it all just fine.
But the actual quality is super dependent on the environment. In poor lighting it completely goes to hell, turning into pixelated goo, and the frame rate drops to the point you can physically see it. It also experiences distortions, so around the edges of objects you get major waviness that changes as you move which can be very distracting.
This is why you'll hear completely contradictory reports of it from different people. If you happened to use it in a really well lit environment with nice uniform lighting you probably would have thought it was really good.
It would be an awful consumer experience of course, but it would be nice just to try and get ready for the real devices.
Let the user attach a pair of mouses (one for eyes, one for hand).
> Next month, Apple will open developer labs in Cupertino, London, Munich, Shanghai, Singapore, and Tokyo
It can be pretty janky and still be more convenient than traveling to one of six cities.
In fact I’d be surprised if VisionOS doesn’t readily eat up all of ARKit dev patterns,
See the block diagram from a WWDC session here: https://twitter.com/firt/status/1665835204953481218
Also the military training applications are potentially enormous. The cost reduction in training with simulated environments is huge, there's already a lot of DOD investment in this. I don't know how quickly this will ramp up, though.
I don’t think Apple spent any time at all talking about games during the launch. This is a general computing device not simple an entertainment one. Nor is it a mass market product like a Quest 2 (yet).
But yeah $3.5k is not a lot to spend for businesses either.
I could see every music producer wanting one as soon as some production software gets VRized. And some novel synth UIs get invented. $3k isn’t much in the Music gear world.
If this is successful it will be the first completely new interaction paradigm since the mouse. Of course so far interactions are mapped down to hover and click because existing software can then run, but the UI tools aren't limited to that at all.
I'd expect that in five years we'll have a completely different sense of interaction with content through Vision (or Vision like products).
(For those who want to quibble -- Touch is somewhat new relative to the mouse, and multi-touch gestures are truly new but very limited. But the design of apps and user interaction hasn't significantly evolved relative to the mouse-bound era.)
What I’m most excited about is to see what people do with the UX.
We are no longer confined to windows that scroll up, down, left, and right.
Application’s now have depth, and the spatial concept will mean the way we view and interact with applications will change completely.
I’m very excited for this.
Perhaps re-calibrate "really well"?
> "very subtle activation motions is a touch beyond gestures"
(We see what you punned there.)
> "tactile feedback to know you have released a button"
With the VisionPro, a click is a touch of forefinger and thumb. That's literally twice the touch feedback of a button.
With a controller, though, you don't have to lose contact with the buttons for activation. Even with a keyboard, where you do lose contact, you feel for the activation of the buttons to do so.
More immediately, though, you also have more fingers for rapid fire activation. Some silly math, but if you type upwards of 70wpm, that puts you at about 400 button presses per minute. Not sure how easily that would translate to gestures, as far as speed is concerned.
Until they sherlock it. Honestly, I'd be terrified to come up with something that became too popular in their app store.
I’m of the mind that it’s a risk worth taking , because it’s an opportunity that wouldn’t have presented itself otherwise
Being too terrified to do something because you might be too successful is just missed opportunity
And established a life-style + employees as well. If it were to suddenly vanish, from experience, you can basically go bankrupt in a matter of months. Maybe its because I've been there (not sherlocked, but a competitor undercut us with unsustainable margins, in fact, they went out of business shortly after we did), and rode that roller coaster. I wouldn't do that again, or go anywhere near that situation. Someone who hasn't been there, would probably be much more willing to take that risk.
And so you have a lot of success and then you make a lot of money during an acquisition.
Sounds like the opposite of terrifying to me.
Yeah, the employees of Karelia really thought the story more worthwhile than their livelihoods.
Stuff Apple sherlocks nowadays are either OS things that have been painfully missed forever and where third party software was a workaround or they implement a baby version of the third party software.
I think their baby versions – while plenty for many – may serve more to prove a market and wet appetite for more than truly making third party software obsolete. Apple doesn’t do niche nerdy stuff but their new features will – and that‘s my hypothesis – often nudge people in the direction of wanting more.
Same here, you're essentially saying you could come up with something massively popular but won't because Apple might sherlock it? The truth is that even when Apple sherlocks things there still exists a market for the "Pro"/"Advanced" version
Reminders: Things, Due, etc
Notes: Drafts, Evernote, Notion, etc
Maps: Google Maps, Waze, etc
Journal (or whatever that are calling it): Day One, etc
and the list goes on. I actually like it when Apple releases a new app in an existing category. It sets the bar and exposes even more people to the idea of such an app, some of those people go looking for a more advanced version.
I’d prefer to compete in an open market. That’s all I’m saying.
You may want to look to Sight reading factory. They do a pretty good job allowing you to dynamically generate unlimited sheet music according to your pianistic capability.
My teacher literally would spend one hour with me on one page of sheet music to ensure I can play the notes evenly with the correct dynamics. He could hear the tiniest hesitation I have, and then come up with specific methods to help me practice that part. That is why I don't think any of such software, AI or not, actually knows how to teach piano, and I doubt they ever will.
Also, multimodal LLMs are improving so quickly that I wouldn't be that surprised if they were able to do what you're describing in a couple of years. But they are definitely not there yet.
I grant that if you can afford a full size piano, you probably don't balk at this thing, though.
I agree that formal training and teaching with a quality provider is likely to be optimal but any tool that adds to the available options for training is worth exploring, especially if somebody already has access to the platform the software is available for - paid instruction is not always an option for many.
We also need to be careful not to gatekeep music by demanding instrument players go a formal route of instruction.
Source: many family members who are music teachers, spanning multiple generations and a wide range of technical prowess.
The Vision Pro is marketed as a device with the potential to replace or significantly augment a laptop and/or TV for a lot of users (I understand there are presently some asterisks needed there). And that's assuming no other novel, high value uses emerge.
Apple sells something like 7 million MacBooks per quarter. Many of them are well more than $2000. I don't think a $2,000 price point is a crazy barrier if the device is able to live up to what it could be.
"Smartphones" weren't the most dominant phone category until after the iPhone.
It's really hard to know until units start shipping though.
It's a strategic release aimed at devs and early adopters, the consumer level device will come in the future at a complete different price point. That's when mass customer adoption might happen.
Also Meta is struggling also because their device is a novelty without many great software. See a pattern?
The Vision Pro seems to be more focused on the end user experience, so hopefully that will help Apple get their foot in the door with consumers. However, I've never used it so I cant say for sure. The company I work at will most definitely be purchasing them as soon as we can though, very excited to either be extremely underwhelmed or extremely impressed.
I don’t even know if it is ability, rather than unwillingness to invest the billions of dollars and 10+ years it would take to get something like this off the ground.
....
You think that developers are going to create a thriving ecosystem of apps when there's no market until several years down the line? Bit of a Chicken or the egg problem isn't it?
Early-Adopter-category developers and buyers will wait to see what the Innovators discover.
It won't be the opening of floodgates, but I think it will be interesting enough.
The hardware is probably being sold near to cost, thus speeding up that side of the adoption cycle. And courting developers will allow them to build up the app ecosystem.
Meta's attempts are currently limited because the sticky use cases seem to be fitness and a little bit of gaming. When you have a device that is viable for some level of work, it's a somewhat easier sell. Apple will be testing this for work (and hybrid remote work) with staff and will have a strong idea as to how viable this is going forward.
I think the incentive is there for employers to favour these (fit more staff in physical offices; and suits remote work well) and that will push it into the general white collar world.
The rest of us will get the apple vision se in 2028 when the territory’s been mapped out
I rarely use my Rift S these days - not because the hardware isn't good - but because the software experience is an absolute dog's anus. Wanted to play Beat Saber the other day. Turned on the PC. 25 minutes later, ready to open Steam. Whoops, now the Oculus app says its out of date. Another 10 minutes faff because what it actually meant was "you need to migrate to a Meta account". Then I had to sit through the tutorials again. Then Steam decided Beat Saber needed updating and took 45 minutes to download 800MB on gigabit fibre. Finally, after just about 90 minutes, I could play Beat Saber.
$3500 to not sit through that absolute omnishambles of a torture ever again? Sign me up.
Best $300 I’ve spent in the last 5 years.
wander works on the quest 2, but doesn't have 3d features in streetview.
However Google recently made the underlying APIs available so it will be soon
Are you sure it's not something else in your case?
Kinda happy they released the iPhone when I was capable of buying that, and this one now that I’m (potentially) capable of buying this.
https://palmerluckey.com/free-isnt-cheap-enough/
Quest headsets are indeed cheap but but perhaps sacrifice too much on the experience quality. People gotta want something before it can get cheap (not my words).
I personally love it, RE4 VR and the month I spent VR boxing every night was some of the most fun I’ve had gaming recently. But there’s limits to that sort of thing, especially which games get ported and VR’d. Apple doesn’t even need a suite of A+ games to get people to buy it, I don’t remember them showing any games in the launch.
I get why Apple pushed it up higher in the market. The first two years will be for developers and very early adopters.
This is like buying an expensive monitor/TV/casual gaming device baked into one.
Hell there’s people who buy $2000 TVs, even though most people buy the $500 ones. This is way more capable than just a new TV.
It's interesting though how counter it runs to John Carmack's philosophy where mass market accessibility is everything. I wonder if looking back people will rethink his contribution as actually holding things back.
It certainly seems like Meta is switching gears a bit with the Quest 3 going out to market at $500. It's going to be phenomenal value for what it is at that price still. But it's going to leave behind a significant segment of people for whom $299 was a palatable indulgence buy or gift for their children etc while $500 is not.
A few hundred thousand users with a massive amount of disposable income.
> Meta, today, has roughly the right price and is working forward to the right device: Apple has started with the right device and will work back to the right price. Meta is trying to catalyse an ecosystem while we wait for the right hardware — Apple is trying to catalyse an ecosystem while we wait for the right price. So the Vision is a device pulled forward from years into the future, at a price that reflects that. It’s as though Apple had decided to sell the 2007 iPhone in 2002 — what would the price have been?
https://www.ben-evans.com/benedictevans/2023/6/15/vision-pro
Same reason Tesla started with the Roadster instead of trying to build a mass-market sedan first.
The potential thanks to the higher resolution is huge. If you ever tried to do any non-gaming task on an oculus, you realize how important resolution is for text. Forget games, this is an enabler for a disruption in user interface.
My biggest gripe to me, let it be either meta or apple, is the vendor lock-in. I'm not going to touch anything made by meta. But apple is not far.
https://docs.google.com/document/d/1F6ns6I3zs-2JL_dT9hOkX_25...
> Apple is not comfortable working under Khronos IP framework, because of dispute between Apple Legal & Khronos which is private. Can’t talk about the substance of this dispute. Can’t make any statement for Apple to agree to Khronos IP framework.
There is probably something that I'm missing here. What is technically preventing Unreal/OpenXR/etc. apps from running on Vision Pro?
Before OpenXR every headset had its own interface. Valve and descendants have SteamVR, Oculus has its own runtime, Microsoft and descendants have Windows Mixed Reality.
Games/Apps targeting these platforms used to have to implement each one of these interfaces separately if they wanted to support that family of headsets. OpenXR shifts the responsibility onto the headset manufacturer, from my limited understanding.
edit: I don't understand why this is being downvoted? If there is a technical detail I got wrong, I'd welcome an explanation rather than Reddit-esque downvotes.
What’s technically preventing OpenXR apps from working? Vision Pro doesn’t implement OpenXR.
I’m curious how spatial experiences will fare.
Also, I don’t like that the gesture tracking in the dj app video demo seems laggy, especially when moving the fader.
But how those SDK events gets translated into UI control movement is obviously going to be dependent on each developer. And I suspect the variance in quality will be much greater than with iOS given everything is in 3D.
Why is the variance any different? You're provided a tuple representing where the user is looking similar to the location of their fingers? You're provided a hook for when the user clicks (with their fingers) in various ways?
This all relies on Apple to accurately capture those inputs at the device and OS level to provide it to developers, which they've apparently done a very good job at. The abstractions on top of this don't feel very different than those that exist on any other platform.
The same thing will happen here except that you have the additional issue of learning how to efficiently render 3D assets in response to user events. After all most developers do not come from a gaming/VR background.
I suspect majority of visionPro apps will be mostly iOS apps with iOS gestures, maybe with one or two 3d gimmicks.
Once Vision dropped it was clear to me that ARkit was always meant to be used with this device, and that most of the iPhone/iPad apps that use it are not the final intended use case. I don't think you can judge Apple's AR success based on the existing hardware.
• AR was an odd proposition for the iPhone, and lidar was an overkill for its gimmick apps, but it was obviously preparing devs and apps for the "spatial computing".
• iPadOS got mouse support, but only with a clunky fat mitten-pointer. This UI makes more sense when it's limited by eye tracking precision.
• Continuity features. Controlling a nearby iPads was a nice-to-have, but for visionPro that doesn't have its own I/O, a seamless integration with nearby devices is a must-have to get work done.
thanks
My personal answer is that I don't know what it's going to be used for, but I'm betting that it will be used A LOT once that use is figured out.
What comments do you want to share from 2007 that will be relevant? The iPhone was released without an App Store and no intention to support 3rd party native apps until developers demanded it. The App Store was released after millions of units had already shipped. A speculative platform of uncertain consumer interest, shipping a tenth of the units in the first year compared to iPhone cannot be compared to a once in a generation phenomenon where the dev platform was pulled out of the vendor by demand.
It will be interesting to see how it turns out but the use cases Apple showed so far are kind of laughable: photos, movies, meditation apps, facetime, 2d games etc. The only tangible benefit was "a big screen". There is more chance these goggles end up in a drawer than an iPhone which was already used compulsively through out the day.
> and the Apple Watch
It's a popular product but are there many developer success stories on the Watch? AFAIK a handful of fitness/media apps are popular but doesn't it otherwise suck as an app platform?
People who spend $3500 on a headset have a lot of disposable income. Perhaps enough to pay $200 for a VR game with good marketing.
The brand + price tag is enough to sell at least a few thousand Vision units regardless of technical specifications.
[0] it's enough for enterprise sales, but they're charging much much more than $3500 one time.
It's especially weird to me considering how nice Metal is to work with, but neither swift not some hacky ObjC/C++ approach seem suited to the task of pushing the envelope of graphics rendering to take full advantage of this hardware. Am I off base here?
Honestly, I hope this takes off is really successful, and kicks off enough investment in competitors so that I can hack on cool stuff in another software ecosystem.
Apple did recently add better support for mixing Objective-C++/C++ to Swift which was more difficult before. [1][2][3] Swift to C++ had more leaps prior to recent updates though it was still possible. Really it just handles the Swift to Objective-C++ to C++ link better.
Objective-C++ has been used since iOS inception to link C++ game engines to Objective-C. All you have to do is compile with g++ or other and typically files with .m are Objective-C only and files with .mm are Objective-C++. Objective-C/C++ and interop with C/C++ actually works really well and every single game engine on the store (pretty much ever game) uses Objective-C++ to link them at a minimum.
We used lots of C++, Objective-C and even Objective C++ (g++ mixing Objective-C + C++, connecting to C++ engine) in a custom game engine that worked on mobile for lots of shipped titles. It is powerful and fun. People have been mixing assembly, C/C++, Objective-C/C++ since the 80s. These technologies so close to the metal and machine code actually work better together than most "modern" frameworks/systems.
A sample of early game engines using this link is Oolong from 2008ish, that was used on an early Quake port and many game studios had something like this for custom engines to get them to the iPhone/iPad/etc. [4]
Fun fact: Objective-C was created before C++. Objective-C all they way back to 1981 but officially 1984. [5] C++ all the way back to 1982 but officially 1985. [6]
[1] https://www.swift.org/documentation/cxx-interop/
[2] https://developer.apple.com/documentation/Swift/UsingC++APIs...
[3] https://developer.apple.com/videos/play/wwdc2023/10172/
[4] https://code.google.com/archive/p/oolongengine/
In more advanced cases when many API commands need to be encoded depending on users' input, and if batching is not enough, you can use things like "MTLParallelRenderCommandEncoder" and multiple queues to take adavantage of multiple threads submitting such commands.
This all means that "pushing the envelope" in terms of rendering is absolutely possible despite the possibly small overhead added by Obj-C - which however also brings some advantages as it often avoids having to call "free" on GPU memory since the MTLBuffers are reference-counted.
Plenty of useful algorithms and GPU computing frameworks but barely anything available with metal backend. Libraries like OpenSYCL, OpenCV, Open3d, PCL don't have any Apple GPU backend.
Anyone here with actual experience with visionOS SDK and its API's, able to give an insight on how it compares to other platforms?
I imagine there's a world of interesting details and differences, considering that the device is designed around a single, very powerful compute specification...
This whole thread is full of people wildly speculating about applications which, without amazing platform level support are each going to be a small miracle in state of the art computer vision applications.
Can they offer full 3d image detection / segmentation to give you a full spatial model of the user's environment? If not, a lot of people are going to be sorely disappointed by the amount of computer vision expertise required to get even basic apps they are envisioning to work. On the other hand if Apple does offer this type of platform support, it will be quite revolutionary.
Meta has offered a range of spatial computing APIs for a while but they have been slow to take off. The Quest 3 should really push this along though. It will be fascinating to watch the two platforms compete on this front.
So Quest 3 will be the first spatial computing device the world has ever had access to that is available at mass market price (<$500) with depth sensor and high quality pass through. It will finally be worthwhile for developers to build mixed reality apps targeted to regular consumers.
https://developer.oculus.com/documentation/unity/unity-spati...
Bahaha, reminds me of the time when I first got the PSVR and my friend comes over. We're playing an FPS demo that came with it (PS Worlds? I think). It's this scene where you're in the passenger seat of the car, shooting up people on motorcycles that are shooting at you. Anyway, I'm standing off to the side watching the TV when out of nowhere, my friend punches me in the face.
I happened to be standing exactly where the NPC driver was sitting in the VR world and my friend wanted to try punching the character.
This first version of the visionOS product line is not the one that will be the 'iPhone' moment.
We are looking at the device that will be ready for these app in the next 5 - 10 years which by that time the device will be smaller that this first version.
Probably going to be called 'Apple Vision' in the form of AR glasses.
Otherwise, it just seems like a confirmation of their reading of the tea leaves.
All publicly available for everyone to see. [0] No inside information needed at all.
Correct either way.
Not unless someone figures out the contact lens or the direct brain uplink.
You mean it’s not a device that’s significantly more expensive and less capable from a feature perspective than the competition at launch (aka the 1st gen iPhone)?
[0] https://www.nasa.gov/sites/default/files/styles/946xvariable...
It bodes well that Kara Swisher seems to really like the device and she's one that has tried all the headsets and been fairly skeptical. Using hands as the primary control seems like a pretty big breakthrough. The default of mostly pass through of the surrounding environment sounds like it also addresses the nausea.
You're right at this price point it's definitely a gen 1/dev device. Gen 3 is where it'll likely get interesting, assuming some killer applications are created.
Having said that, I haven't met that many people that get nausea from wearing sunglasses, so at the theoretical limit there is some level of quality that seems at least almost universally accepted.
As an ex Daydream (R.I.P.) user, the app store screenshots look oddly familiar. Maybe Google can resurrect Daydream (probably under a new name, or just call it Daydream Pro?) and reuse some of their sunk investment as software for an actual VR/AR headset rather than a thingie where you have to insert your phone?
- Night vision app using the truedepth and lidar sensors
- Drone/robot/telepresence HUD. It would be incredible with stereoscopic cameras
The drone things, amusingly, are already price competitive. At least once you factor in the cost of the drone.
That's interesting, I could see this as part of the security task force for world leaders and such, some extra guys with virtual eyes spread all around and can watch people around corners.
Mark your calendars!
The device itself is running the base M2, so there's reason you can't right code using a device with the exact same chip. I would recommend at least 16G of RAM, with 32 being more future-proof. Rumor is the device has 8 on-board.
This is different territory compared to marginally useful 2D app that millions of developers produce.
To make a believable virtual/3D experience requires enormous investments if you don't want it to look like a video game from 25 years ago. It's quite telling that Apple rushed through the 3rd party demo session at the event, as every example looked terrible. As terrible and useless as 99% of AR apps produced thus far.
I would suspect that only giants like Disney could produce a "killer app" experience that woes people and make this worth the money and discomfort.
And on top of that, yes, lots of developers will produce what are basically 2D apps where they take an existing app and project it in space, possibly showing multiple sub screens. Slightly more useful if done well, at best. Financially likely the only realistic scenario for most developers but it also cheapens the appeal of this device.
Not to mention that you need to apply way more real world scrutiny to possible scenarios. As an example, somebody suggested a cooking app.
To solve what problem, exactly? Cooking is instructions and timers. Yet with this device one can project more information. What information, exactly? Cooking requires free movement, to grab items, throw things in the trash, but here you are with a thing strapped to your head with low visibility. Possibly wired as you need to be thinking about battery during a long cooking session. Maybe you're cooking with two people, as couples sometimes do, then what? Countless real downsides, merely theoretical upsides.
Don't get me wrong? Will the tech elite embrace it and will a few influencers rave on about this cooking app on their channel? Sure. But that's not my point. My point is that you need convincing apps with tremendous added value to offset the price and discomfort of this device.
Without that, it will be solo movie watcher pro.
I agree that there are use cases that when you think them a through a bit just aren't as practical as one thinks, such as the cooking scenario. Walking through scenarios in your head and thinking through the logistical issues with using such a device is a good way to ground yourself in some of the VR hype, in my opinion.
A developer doesn't need to make a killer app, just supplementary apps that people will use in this environment. And yes, some people will just make money making sure their 2D iOS app works in this environment, and a subset of that will convert it appropriately to work as a volume with depth.
In this environment, Apple handles the difficult spatial stuff and many developers will just build planes or volumes within that, and get solid mileage from that.
As anyone who's arrived early at an ecosystem knows, you first get used to the basic stuff, try a few free gimmicks, and then you're looking for apps to try and buy. Anyone in that marketplace at that point has far easier opportunities. Leave expensive-to-develop apps like VR courtside experiences to NBA and whoever else - those will sell the device. Smallfry developers will be fine otherwise.
I wonder how long before people start bumping into lamp posts because they are distracted by one of these.
we're still at the apple newton stage
maybe in 10 years we'll be there