Disney Research unveils augmented reality books | Hacker News Reader

Disney Research unveils augmented reality books

disneyresearch.com

162 pointsdccoolgai51 comments

This implementation looks amazing but also seems practically very awkward.. I can't imagine children being so patient and accurate like in the video!

We've been doing things like this for years at the HITLabNZ. Check out our latest project, which is free this week! http://colarapp.com

Just tried this in the office with my phone. Impressive! You should submit this as a Show HN - really deserves to get some reviews.

Good fun for those with kids and even mucking around in the office demoing to friends.

You should throw some more money at this to get a few dozen more models/pages going. I can't think of a 1-5 yo that wouldn't like this. Colleague with children said "My kids are going to lose their shit when they see this."

How do I do a Show HN? Just make a post?

Yes, but if you prefix the title with 'Show HN:' it's considered better form when its your own work.

Yes. Click submit up above and then provide a title and a url.

Wow, the demo video is really remarkable! I will try it out with the office iPad tomorrow. Really great job! The video speaks a thousand words even without any spoken words. =) How do you align the coloring book pages from the camera feed? Do you detect the corners of the printer paper or is there some printed, hidden marker on the page?

I'm not involved with the app, but there's no need for a hidden marker in this case -- the page IS the marker. It is a planar surface with one of a few known black and white images on it. So: Detect features in each blank drawing, store them in a database included in the app, then use the same corner detector and descriptor at runtime and you have a very constrained feature matching problem.

If you're not familiar with feature detection methods, SIFT is a famous one, though there are much faster methods existing today: http://en.wikipedia.org/wiki/Scale-invariant_feature_transfo...

We use Vuforia and Unity. Adrian Clark did his PhD thesis on tracking and we based our app on his ideas and work. We don't use SIFT as it is patented but we have better research in the pipeline.

It appears to only work on certain pre-defined coloring pages.

Naturally. Those 2D print pages must have 3D models in the app (right?). I'm wondering how the app does feature detection from the camera feed.

Our 3D modeller Shun spends about a week per page carefully designing and animating the models. You are correct, they are per page and completely pre-defined. The fun part is colouring in the pages and seeing the results :D

Even me, 28 year old male, can have a lot of fun with this app.

Why don't you guys allow for printing from within the iOS app?

Personally I'd like to do that but we have a lot of limitations with the Unity platform and supporting multiple platforms. Every feature needs to be implemented at least 2 times (iOS, Android) and tested.

Perhaps in the future once we get some money coming into the company. Right now we don't have any more funds left for development.

I'm sure this has already been discussed, and I know this falls under "easier said than done", but you guys should really try to get this on toy aisle shelves. Perhaps in the form of a book? Maybe have a prominent "Try me!" feature on the cover instructing the user to install the free app and point it at the cover to get people pulled in.

Yeah, other companies are working on this too. I think we are in a unique position at this time with what we've developed.

This is a great idea. I could see this becoming a very popular coloring book series. I'd buy it for my kids.

We've been trying to get major publishers interested for over 8 months. Molasses comes to mind.

With the right marketing and execution and a little luck you'll have tons of traction and be in a better position when they come to you to license your tech for their books!

Why would any kid ever want a "plain old" coloring book without the 3D augmented reality?

Very cool—my kids are going to love for sure!

The demo of the app looks really fascinating!

I tried installing colar app on an Android 2.3.6 (Samsung GT-S5300) phone, but it seems it's not compatible with that device. It's a low-end phone, so I guess I will need something more powerful.

Since Google Play doesn't display the device compatibility anywhere in the store (besides "This application is not compatible with your device") could you please state the application requirements somewhere on your pages?

Sorry about this. We are still figuring some of this stuff out - you will need a modern phone with a decent camera at the very least.

What a fantastic app! Feels like a reverted foldify or so :-) I've spread the word, I'm sure you should see a rise in downloads today from both Switzerland and US. Thanks for sharing this!

That really is awesome. I just shared it on my Facebook page, and there were people talking about downloading it within minutes. Seems like marketing this thing should be relatively easy.

Haha, if only!

Thanks for the link. Had this installed on iPad and 5 yr old daughter can't stop coloring the pages and watch them come alive! Well done.

Thanks!! One of the most rewarding experiences I had was watching children at one of our recent workshops - when they first see what is going on their faces light up and it is amazing to watch.

That's an awesome app! How is the marketing going? Because I think this could be a really big hit.

Well, I think marketing is the single biggest challenge that we face. People who have the power to make the decisions don't always understand the technology.

I saw you got press in TechCrunch today, congrats!

Yeah, it is freaking amazing and the first paragraph blew me away.. I'm hoping we can keep it up over the next few weeks.

> which is free this week!

What do you mean free this week? What happens after this week?

You obtain the software through the app store/play store, so I'm assuming that after this week the software will be set at a price >$0.

Yep that's right.

i.e. it won't disable the existing pages?

Nope.

The in-app purchase is currently set to $0. At the end of this week it will be set back to whatever it was previously, a few dollars.

When I tap "unlock pages", I just get a blank screen titled "unlock pages". Is something wrong?

Yeah, it might be a bug. I've seen this before - we are using standard Unity plugins but honestly I'm not confident that they are really reliable. Try restarting the app.

If you find this stuff interesting, check out TEI, the conference at which this was published:

http://www.tei-conf.org

It's an amazing conference that has been going on since 2007- great people, much smaller than CHI (of course its scope is much narrower, and there is a lot of cross-pollination), very high quality and diverse publications. Even though it's an academic conference, it has more of a hacker/DIY feel to it than CHI in my opinion, which is what I love about it.

Speaking of which, the deadline for TEI2014 is Aug 1st... back to writing I go :)

(and if you're into this, please contact me through my profile! Tangible/embodied interaction is a small world, would love to hear from fellow HNers who are in it)

This looks pretty similar to Jeri Ellsworth's CastAR project, except not head-mounted.

http://makezine.com/2013/05/19/jeri-ellsworth-unveils-augmen...

Making the markers visible in the infrared spectrum is genius.

For books and many other print media however, it's just overkill. You can do it markerless just by tracking known content on the page.

It's great for tracking within the whitespace however - that's where it's a cunning solution!

Still waiting for proper 'ractives.

an iPad could do it now, but we don't have the proper capture/reprocessing frameworks in place (outside of research).

I was just thinking how close we must be to "A Young Lady's Illustrated Primer" (see: "Diamond Age", by Neal Stephenson, referenced by me and seiji here. GREAT book.)

Slow start, insane end, typical Stephenson.

But do we know enough to tell software how to raise a child?

There were two versions of primers. The mass market version had auto-schooling (giant boat of threatening asian girls). The expensive ("for royalty/CEOs") version had a live adult person interacting with the child, but the child didn't know a real person was behind it. The child just thought the book was very good at being instructive. The actor of the expensive primer had no way to communicate with the child except through Primer-Approved stories, but the actor would direct and personalize the stories as necessary for the child's life conditions (i.e. "remote parenting").

Yes, I forgot. Her "other mother" - thank you. Including the cool escape from abusive home section.

What a great book.

Did Lucas ever license the rights to anyone for the futuristic chess game on the millennium falcon? Seems like an obvious application of this tech, particularly now that Disney owns the franchise.

I think we can see the next mobile hardware killer app.

I mean the number of cool "what if you did this" reminds me of early days of GPS and indeed GPS in phones

This is similar to some of the AR games on Nintendo 3DS, with the exception that you don't need to look through a screen to get the effect. You can imagine this on Google Glass too, viewable only by those wearing the HUD.

They ought to head mount that thing so it follows wherever you are looking.

Reply on news.ycombinator.com