AR.js: Efficient Augmented Reality for the Web | Hacker News Reader

AR.js: Efficient Augmented Reality for the Web

github.com

226 pointsuptown34 comments

The WebARonARKit repo is simply a tiny little iOS app that make a WKWebView, hooks up a few message handlers, fires up ARKit, and sends messages back and forth, by the book.

But the WebARonARCore repo is a whole freaking clone of the entire Chrome browser!

Isn't there a much simpler way to extend an off-the-shelf Chrome component like they did with WKWebView on iOS, without rebuilding the entire honking browser?

I know there is a WebView chrome component for Android -- why didn't they simply use that, the same way as they did with WKWebView on iOS? Performance? Does it require features the WebView Chrome component doesn't support?

Yes, they could have done the same thing with an Android WebView. But the Chrome team is actively working on implementing WebVR (which also want to support many AR use cases) so they already have a native (= higher performance, lower latency, etc) implementation to modify with some smaller changes to use the ARCore API.

anyone else have issues trying to get the demo to work?

This site (https://jeromeetienne.github.io/AR.js/three.js/examples/mobi...) on my phone is telling me that my webcam can't be found. :|

It asked me for permission, I gave it, and it worked really smooth. No problems here.

On iPhone it theows an alert saying navigator.mediaDevices not present

I’m getting the same error, and I’m on the iOS 11 GM(not the public release yet).

iOS 11 is out. Both the GM & release have the same build 15A372

Thanks. I wasn't sure if it was a different build.

That's a web rtc problem. The library doesn't have a cross browser compatible adapter for it.

make sure you open in the full safari and not the in-app web view

For it to work on iOS you need to update to iOS 11, that may be the problem.

ah, that does seem to be the problem, at least for me. Thanks!

So what is suppose to be happening when you allow camera access? Other than simply recording

Isn't the main selling point of the current AR iteration, that these markers aren't needed anymore?

There are still good reasons for markers, like syncing two different devices to the same augmented space.

Does anyone know if Apple's ARKit has a solution for this?

That's what I was hoping to see from the game demo during the keynote, but they didn't allude to it. That's the next key element we're going to need.

not yet. right now ARKit feature points are not 'persistent'. Meaning, they are only really used to help track 3d space frame over frame (with the internal gyro). Getting a 3d map that can be shared with multiple people is another layer of complexity not yet integrated into ARKit

No, the toy AR in smartphones doesn't really know depth or scale to the accuracy of a known marker. They just make assumptions and do basic plane-finding to avoid collisions. You don't have to know depth to make a hot dog sit on a plane, you just have to know it's a plane and how many pixels tall you want the hot dog.

It will be a long way before markers are obsolete. My guess is about 5 years.

Have you tried the "toy AR" in new Apple devices running iOS 11? it's quite impressive how accurately it's able to track most environments. Sure, having a known marker makes that process a lot easier, so you can avoid the issue entirely even where ARKit has problems (a large featureless surface, for example)...

But even with the current limitations, it does feel like the next evolution of AR -- one you can just start using with no setup. Printing a marker is not something a casual person, just interested in checking out a new app, would do (assuming they have access to a printer, or a second device, in the first place). I think that it's a different ball game, and agree with OP that it's the main point of the new generation.

That's not to say anything negative about this project specifically -- it's awesome, and really cool to see something like this perform well on the web in general, and mobile web in particular.

Yes, I develop monocular slam for a stealth startup, so I have directly dealt with the requirements for markerless AR. It's a well known fact in SLAM that a physical measurement is necessary to determine scale. IMUs get close, depth sensors are better but have rough resolution, seeing a known object (like a marker) in the space is best.

Because the toy AR displays on a small 2D screen, there's a number of shortcuts that can go mostly unnoticed.

Google recently added ARCore and ARKit support to an experimental Chromium build [1]. I'm awaiting this support getting mainlined into Chrome.

[1]: https://developers.google.com/ar/develop/web/getting-started

That's really interesting!

They didn't add ARKit support to Chrome. ARKit only runs on iOS. Chrome isn't allowed to run on iOS. (The iOS Chrome app is just WKWebView with a Chrome branding wrapper, plus a few features bolted onto the side.) But even if somebody did build the real Chrome browser on iOS, Apple wouldn't allow anyone to publish it on their app store.

On iOS, WebARonARKit simply uses WKWebView's standard method of sending messages back and forth to the browser:

[userContentController addScriptMessageHandler:self name:@"WebARonARKit"];

https://github.com/google-ar/WebARonARKit/blob/master/WebARo...

My question is why did they have to make a custom Chrome build on Android, to simply integrate the browser with a library like they did on iOS (WKWebView <=> ARKit)? Why didn't they use the off-the-shelf WebView browser component for Android instead of building their own monolithic custom Chrome app, just to integrate the browser with an Android library (WebView <=> arcore-android-sdk)?

This is super cool. I really appreciate the ability to use (albeit a more primitive version of) this technology on my iPhone 6. It isn't even 4 years old, and it seems to be one of the largest iPhone groups in terms of device share next to the 5S.

When iPhones 8 and X came out, I was thrilled more so with iOS 11 featuring ARKit, but bummed out my A8-powered device could not run their advanced augmented reality software. The price points are high right now for me, and what the devices provide are incremental, making me wonder when smartphones will taper off like PCs did in terms of delivering power across new generations of chipsets.

For the time being, software like this allows hobbyists like me to play around with concepts I'd love to but are otherwise locked out of for now. Thanks for sharing!

Even when devices are capable of running some software, I'm wary about it. I've seen enough VR/AR turn phones into a brick of magma.

yeah, star walk was doing ar on iphones since 4s iirc, it stinks that arkit can't do it

This depends on ARKit for iOS, so it won't work on any phones that ARKit doesn't support. On older phones, ARKit falls back to 3DOF "rotation only" mode that is not anything you can't already do in WebKit with DeviceOrientationEvent.

https://developer.apple.com/documentation/webkitjs/deviceori...

Nice work, With huge companies picking up AR and VR and making them more mainstream, I think soon we will see more developers in these fields and finally some very useful applications.

Even if it doesn't deliver everything ARKit does, this seems like a great way to promote an AR app or brand on the web.

Quite cool, congratulations.

I am not always positive about WebGL stuff, but this one run quite well on my devices.

I think the project is cool, but with Google recently putting real work into this effort[1] I don't see how this project is going to succeed over the competition in the long-run. This is hard stuff. Marker-based AR is not enough. And as far as I can tell from the creators twitter account, he doesn't seem to have the necessary math background to tackle the research heavy papers needed to implement the really cutting-edge stuff. Hopefully some computer vision PhD can help out otherwise I would bet my development time on the stuff Google is creating.

[1] https://developers.google.com/ar/develop/web/getting-started

So sick and tired of people who act like only Google/Facebook/etc. can do cutting edge work, or that single-contributor projects aren't useful.

Here's something you probably don't know, since you're so quick to dismiss AR.js. AR.js uses Google's WebAR where it is available. So no, WebAR is not competition for AR.js, anymore than Native ES7 support in Chrome is competition for Babel.

Google doesn't have a great reputation for long-term support of projects that are outside of the company core business.

Reply on news.ycombinator.com