Apple enters the augmented reality fray with ARKit for iOS
techcrunch.com
techcrunch.com
What's fascinating is Apple has a large team focusing on HCI & AR - what "pinch-to-zoom" and multitouch did for smartphones is what Apple will need to successfully pull off the "big bang" for consumer AR.
I'm curious, what have you read/seen that makes you think this unknown product is delayed before being announced?
https://www.reddit.com/r/apple/comments/6ezhwm/iama_foxconn_...
Rough bill of materials of Project Mirrorshades - Apple Iris. Note Iris is Siri backwards. Kopin NED Acetate frame Polarisized or prescription lens with Zeiss smart optics Bone conduction modules Microphones (noise cancellation) Light sensor Accelerometer for step tracking and head movement app navigation Magnetometer for navigation Capacitive Pavel Ceramic battery Apple chipset Charging circuit BL5 Induction module
Crystal, champagne and black. We only saw transparent reader lens but I understand they can procure polarised or prescription lenses Cellulose acetate Injection mold frames into an aluminum mold. Colors are added and tumbled for finish. P3 frame design
Two sizes, men's and woman's
Usually 4x the BOM cost; it would indicate around $600 mark. Crystal is just clear. But clear without artefacts is difficult.
I work closely with people in the AR industry and everyone knows Apple has been investing heavily in developing this wearable device. Augmented reality is the next major fundamental platform shift, and Apple is hoping it will play the role the iPhone did as the catalyst for the smartphone/mobile era - but this time for consumer AR.
What Apple announced today is simply the beginning - they are providing tools so they can establish the foundation for "the largest AR platform in the world" (as mentioned at WWDC today).
Side note: There's a video from the 90's or late 80's showing an Apple concept for augmented reality glasses - it features a woman translating French or something while reading a restaurant menu. I can't seem to find it but it was on YouTube at some point...
This also creates an opportunity for Apple. It's not just about the technology. You need to really figure out what customers will love and make the right decisions on what to include and what to leave out.
Smart glasses have never been "cool", but keep in mind that prior to the iPhone smartphones were generally uncommon for average users - unless you were a geek (I used to use a Palm Treo/PDAs in HS).
I think what they're aiming for will be something like the AirPods but intended to be worn all the time like Apple Watch. If AirPods were slightly smaller and had all-day battery life or wireless charging they could stay in your ears all day as a "wearable", like how many wear hearing aids today. AR Glasses would have to be lightweight and as un-intrusive as possible - they'd have to appeal not only to people who wear eyewear regularly but also to users who are not accustomed to doing so.
Perhaps they'll incorporate a retinal projector like Magic Leap or some other novel tech...
How would they track objects in 3D without a camera? Accelerometers are not accurate enough to to overlay-type augmentation they demoed. Unless you mean that the user won't be able to record pictures/videos so they won't call it a camera.
Still - an AR device without a camera?
The same leaker said there were also plans to release new styles seasonally (presumably in order to stay in style) – which begs the question: would the projection apparatus be modular?
FWIW the "leaker" claimed the project was postponed indefinitely due to battery manufacturing feasibilities.
https://www.macrumors.com/2017/03/27/apple-stepping-up-ar-ey...
- Probably using a lot of Metaio tech, a leading AR dev solutions provider Apple acquired/shuttered in May 2015
- Unfortunate naming. Sounds extremely similar to ARToolkit [1], the leading open source/free AR dev solution.
- Markerless is the future of AR, so good on Apple for getting this markerless tech out to compete with Google Tango and Microsoft Hololens. The status quo is currently marker-based recognition using "tags" or QR codes. It's also much harder than marker-based, or "on body" (Snapchat, Facebook AR Studio) recognition. I'd say Google Tango has some of the most impressive SDKs out there for tracking, area learning, and depth perception, but it does require specialized smartphone hardware from Lenovo or Asus, which significantly limits its utility and mkt penetration at this point. Google was just too early with Glass and got burned, but it needs a stronger hardware platform, which is something that Apple can deliver.
- The Unreal-engine AR video demo was cool and the graphics seemed decent for real-time rendering on an iPad, but the real future of AR applications (read: $$$) will come from industrial applications on wearable devices or head mounted displays such as the Hololens, Vuzix, Daqri, or Epson Moverio devices. Examples of industrial/enterprise AR applications: Remote help, complex assembly, pick & pack, line monitoring, materials handling, systems training, etc. This is the market Microsoft and PTC (Vuforia, ThingWorx) are targeting, not the consumer gaming/advertising markets, which use more "basic" forms of AR. This is not to say Pokemon Go/Snapchat do not generate a lot of revenue, but it's very debatable if they can be considered true AR applications.
- Other vendors in the AR development solutions space include: Aurasma (HPE), Blippar, Catchoom, EON Reality, Kudan, Pikkart, Wikitude, among others.
- In conclusion, Apple is currently stuck between a rock and a hard place. It's somewhat late to market with ARKit, will have to compete with Facebook, Google, and Snapchat for consumer AR-oriented developer mindshare, and will have no play at the industrial/enterprise market if it limits its tools to development for the iOS family.
- The best way for Apple to alleviate this jam is to release its own AR glasses/headset, which it is widely rumored to be developing.
See also portable music, phones, tablets …
It's Apple's SOP, and if they've shown anything in the last 10-15 years, entering markets late is not a barrier to them competing
I'm not sure this line of reasoning holds up. You mention 3 markets:
1. consumer
2. enterprise
3. the "market" for developer mindshare
Apple doesn't especially care about being early or late to any of these markets. Let's take them one by one.
1. There is currently no real consumer market for AR, so Apple is not late. And I don't see any reason why Apple would not be content to wait and see if one develops. It's not clear how releasing an improved version of Google Glass saves Apple from any kind of "jam". People are not clamoring for this type of product, because its value is highly unclear.
2. There is currently no real enterprise market for AR, so Apple is not late. Apple has no history of making industrial equipment. Are you saying Apple is about to take the unusual step of starting a whole new sales channel, targeting a whole new class of buyers, to try to get first-mover advantage in a new and unproven market helping warehouse workers do line monitoring and "remote help"? It seems unlikely.
3. Apple has a huge, strong developer base it is in no danger of losing. Again, it's not clear how offering an AR SDK gets them out of a jam.
If I had to guess as to why they are playing in this area, it may be to help seed the ground with more iOS devs familiar with this tech, so that in the event that the market does develop, they are in a good position to harvest ideas, talent, and apps.
2. Enterprise/industrial AR market - wearable AR-enabled applications - I've already listed them. This market is very small currently...but it exists. I've talked to engineers and AR tool vendors in this market, creating applications for Vuzix, Epson devices. My argument is that this portion of the market will see a lot of growth over the next 5 years. Hard to explain Microsoft's play with Hololens, or PTC's ThingWorx Studio tracking tech away.
3. They are in a jam with AR application developers specifically, not all developers as a whole. You're right they have a large, strong mobile developer base, but AR developers will move away from mobile/tablet-focused platforms (iOS) to Android/Windows in order to develop for AR wearables.
They're playing in AR because they're looking for the new iPhone...a new form factor to deliver new applications and jump start sales. AR glasses could be that form factor if they can execute like Jobs did with the smartphone.
Apple is doing very well for these applications, so again they're not behind.
> Enterprise/industrial...This market is very small currently
Exactly.
> AR developers will move away from mobile/tablet-focused platforms (iOS) to Android/Windows in order to develop for AR wearables.
Why are you so sure they will leave mobile? Wearables have three benefits: 1) hands-free, 2) immersive, 3) Stereo vision. None of those features have shown to be required for AR killer apps. So far the leading apps (Pokemon Go, SnapChat, etc) have not needed them. Do you have some reason to speculate that those specific benefits 1-3 will grow in importance?
Hands-on/non-immersive is actually a feature in some ways, it allows the user more control over their degree of immersion. And head mounted overlay AR requires a massive increase in processing power in order to get acceptable latency. Latency doesn't matter as much for handheld passthrough AR. Lots of tradeoffs there, it's not clear to me HMDs are a win for consumer applications. Industrial; sure.
We've seen some of the stuff Microsoft talks about with hololens. The idea that it could be useful to some people in industrial/business settings.
What if instead of needing a Hololens dev kit for $3k or waiting for a production version that will probably be at least $1k… you could get your application to work with the $350 iPad when the new OS comes out in September?
Sure Hololens Will be better at some things, but that's a hell of a price difference and it will be available retail soon. By the time your app is ready for your workers to use iOS 11 will probably be out.
Almost certainly - Metaio presented realtime lighting estimation at InsideAR (their dev conference) in late 2014. It's also been shown by Hans Kaufmann's research at TU Wien in Austria [1] although this has tighter links with other vendors than Metaio.
| Other vendors in the AR development solutions space include
I've had the pleasure of dealing with most of these folks within the last few years. They all have their own specialisms around certain tracking methodologies, pricing models and other features. Evaluate them with your own purposes in mind, and choose the right tool for the job.
| will have no play at the industrial/enterprise market if it limits its tools to development for the iOS family
In this market, Vuforia really does have it fairly cornered - it seems to be a chunk of what PTC are chasing, although they're far from turning a blind eye to B2C.
Edit: Grammar fail.
[1] https://www.ims.tuwien.ac.at/research/virtual-and-augmented-...
I have been avoiding building AR for Unity for the reasons you described xD!
The only problem is that I very much believe in the web application model as a means of reaching a wide audience with the fewest resources, but Unity's WebGL export is functionally useless.
You should take a look at AR.js, its a cool project that will hopefully provide that wider accessibility. The challenge with Apple's ARKit is that its AR technology( is much more challenging than Vuforia, and tricker to implement.
It's not exactly a small market to address. Square has shown that you can get people to buy apple hardware for used in some specific capacity.
Currently locking the AR to the real world requires a QR code-like mark but they are demoing AR with no mark. Not having a mark requires 3D data of something in the field of view:
https://www.youtube.com/watch?v=-S-4oJ8C_fo
skip to minute 19 to see the markless demo
Nope just a kit for developers to build games only or does the ARkit allow access to the camera app? I'm dying to see innovation from Apple like in the camera app when you open it there's an option to view how things look in the past(100 years ago what did this spot in New York City look like.. where you can take a pic of yourself in the past).
I'm sure there's oodles that can be done with an AR Camera.
Apple and the others to me seem to be slow to innovate.
What will happen is a developer will build the app I mentioned above (could exist already) and then Apple will later build it into the camera app. I've never seen an app maker build functionality into any of the iPhone built in apps.
What you _can_ do, (and it's quite easy), is to create your own app, capturing video from the camera. Then you can overlay any AR content you wish onto it.
This allows developers to invent future "camera" apps that could be better to use than the regular camera.
- Time Machine(idea noted above)
- Daylight View (if currently nighttime)
- and other AR apps devs or Apple creates.
And what would be the point? You can just launch a 3rd party AR app and it will use the camera APIs to show you the live camera feed, exactly like the built-in Camera app does.
As an independent developer, I hope the AR kit will make it much easier for me to create something fun and possibly integrate with Siri and other kits. This is what I'm hoping for :D
I know VR is hot right now, but I've always had the thought AR > VR.
That's why the little bit during the demo was interesting. They showed Pokémon go using AR but it was actually doing it correctly, where the Pokémon stayed in position as they moved the camera around.
I was wondering what Pokémon Go would do on android. Will this feature be iOS only or will they implement their own version on android?
I'm sure part of it is just the ability to do useful things, part of it may simply be that they have such had room on the chips that they can do it without draining battery life horribly. I remember those old apps clearly used a lot of processing power.
Tango takes the tracking to the next level. Maybe future iPhones will have that kind of IMU and other sensors, allowing Apple to expand their AR API.
I tried Pokemon Go briefly, and their "AR" was a complete joke compared to this.
(the last one being released just a few days ago...)
Of course there are a lot more resources out there and a lot of the cutting edge research is involved with deep learning techniques (starting with e.g. https://github.com/alexgkendall/caffe-posenet).
You may also want to watch out for this conference http://cvpr2017.thecvf.com/ scheduled for the end of this month and is usually a good source of cutting edge research...
https://developer.apple.com/documentation/avfoundation/avdep...
> Camera: Updated video and image compression, improved Portrait Mode with support for low light, plus a new Depth API for developers.
from https://venturebeat.com/2017/06/05/apple-announces-ios-11/
So you can now access a stream of depth data. I'd put money on ARToolkit using this stream when it's available. For now, it would only be from the iPhone 7+.
[1] https://developer.apple.com/documentation/avfoundation/avdep...
Perhaps that data isn't granular enough (or fast enough) to be useful when doing AR compared to the normal camera picture plus the motion sensor data.
Facebook made a big deal in F8 out of adding cameras to all its applications. They're in the AR game big time.
https://www.nytimes.com/2017/04/18/technology/mark-zuckerber...
Microsoft has the HoloLens, which technology they have also cannibalized for a series of headsets they are dubbing "Mixed Reality" for the depth camera pass-through they are performing.
HTC is supposedly developing their own AR/inside-out system.
Leap Motion, Intel, Qualcomm, Meta, and Magic Leap are still considered in the running, with varying levels of believability.
And then there are a slew of different companies that are doing AR solutions with either the standard webcam (e.g. Facebook, Snap, Vuforia, Wikitude) or their own, standalone camera (e.g. Zed, Occipital).
I have no love for Google. I certainly don't trust them enough to purchase a product they make for fear of them killing it. Plus they mine everything they can get access to for who knows what ends. "So for those reasons, I'm out".
MS.... Meh. No one has come to expect great things from them.
The market will decide who wins though.