Microsoft Reveals the Science Behind Project Natal for Xbox 360 | Hacker News Reader

Microsoft Reveals the Science Behind Project Natal for Xbox 360

scientificamerican.com

38 pointsstakent23 comments

I worked in computer vision for my MSc three years ago, and based on the state-of-the-art at that time I would be surprised if Natal going to be as robust as the mass market expects. This kind of thing is notoriously hard and error-prone. The information coming in from cameras is often so noisy and incomplete it is very difficult to tell what you are seeing.

When Natal was unveiled at E3, it was having exactly the kind of issues I expected: http://www.youtube.com/watch?v=NKtpJW9pg84

Also the rumour that they have chopped Natal's onboard CPU and moved most of the processing to the Xbox 360 CPU does not fill me with optimism... Prove me wrong, Microsoft, prove me wrong!

Makes me think of the Newton hand writing recognition. It was pretty amazing for its time. But the errors made it worse than a keyboard for most people (as decided by the market, at least). Natal could correctly recognize 90+% of all motions correctly, but if it occasionally makes a mistake that causes your character to die or the opponent to score a goal on you, the public might stick with their Wii-motes for now.

I got to play the Natal Ricochet game this summer and I must say that it worked flawlessly. The only thing that didn't feel natural was the response time, which they said they were trying to improve.

Maybe, but performing well in controlled press tour and demo environments is very different than performing well in a million different kinds of living room and lighting configurations. I'm not saying the tech is going to utterly fail, but these things typically do present a lot of problems for computer vision technology.

Sorry for the obnoxious link, I had sound off when I posted it! Here is something less headache-inducing: http://www.youtube.com/watch?v=DeZep21f32o

Yeah, skeletal tracking is an open research problem. It's hard. Very hard. But believe me when I say, the tech has come a long way since that demo...

well, BAM! here it is

One thing I want to see done is a good swordfighting game. The Wii hasn't achieved that yet, but some of the videos for Red Steel 2 show promise. Overall, it seems that even with good motion sensing for swinging the sword, footwork is still an integral part of the whole experience, which the Wii can't manage. So in this case, perhaps Project Natal will be the first to manage it?

Then again, perhaps rather than wait for a game to simulate it, it would be better to do the real thing instead.

The problem with that is twofold. First, the Wii doesn't support absolute 3d positioning. So it can't map an in game object 1:1 with the controller. From there, it is left looking at your acceleration data and then after the fact matching it to a predetermined set and playing the move. It becomes a weak substitute for pressing a button.

The second one is how to handle tactile feedback. Ideally, when your sword hits another sword, you'd want the player's sword to stop. You might be able to vibrate as a cue to stop moving, but hard to say how that would work. You could work around it by having the on-screen character tween back to the real-life position after a period of time, but it still leaves a lot to be desired.

Long way to go before you can sword fight realistically in a game.

It can with the new add-on thing (the wii motion plus or whatever it's called).

Actually, it can't. That adds more precision to the acceleration measurements, but it still can't do proper absolute positioning.

Please do not go out and pick sword fights.

Natal definitely has potential to revolutionize HCI, but I find it hard to believe it will help Xbox match the success of the Wii.

I cringe every time I see that video from E3 of that poor woman flailing on stage, swatting virtual dodge balls. Like Tony Hawk: Ride, when games start becoming too real, they tend to stop being fun.

Also, consider amputee or wheel-chair bound gamers who don't fit the typical body shape. Perhaps a small market, but a potential PR nightmare when Iraq veterans are told they don't have enough points of articulation for Natal.

I wonder if this is a tactical error. While the Wii sells well, there really isn't a lot of compelling software for it. Often the interface adds little, or even takes away from gameplay. Perhaps it's just really hard to get right.

disclaimer: I own a Wii

I think that the fact that Natal is coming out several years after the 360's introduction will help. The Wiimote/accelerometer-based-control has been marketed as an integral part (probably the most recognizable part) of the Wii experience. In keeping with that, most people don't own Gamecube-style controllers, so making games that use a more traditional interface is a losing proposition on the Wii.

In contrast, Natal is a purely optional add-on, and I'm not sure on this but I'm not sure it will be possible to control non-Natal-enabled games or the Xbox 360 dashboard without a traditional controller. This way, game developers can choose the controller paradigm that makes sense for the game, and the Natal ecosystem can grow organically, rather than have it foisted upon people. Maybe this is a recipe for languishing and fragmentation, but I can see how this strategy could work out better for Microsoft than Nintendo's did for them.

The technology could also be intrinsically much better, who knows?

Optional peripherals are a chicken-and-egg problem. Consumers won't buy it if there are no compelling games for it, and publishers aren't willing to commit to the peripheral (e.g., controllers not allowed) if there are not enough consumers that own the peripheral.

Microsoft can address this by stepping up and making Natal a required accessory for some hit game. The trick then is finding Natal's "killer app", which we have yet to really see.

It's going to be tough - see Microsoft's FF Steering wheel and the 360 webcam as evidence. The wheel is confined to racing games and isn't useful for much more, and the webcam is fun for a few nights but doesn't have a lasting appeal. The only webcam game I'm aware of was the one shipped by MS, and it's not very fun (and very tiring on the arms). Both devices are successful in their own scenarios, but are still niche products (how many games have you seen that use the camera?).

I'm not sure if the Wii controller is accurate enough to realize what imaginative developers would have in mind for it.

Case in point: Force Unleashed has a very limited set of motions (vertical/horizontal swings). This game should have been the killer app for the Wii, but it was really awful.

Natal proposed to have the accuracy and range of motion an imaginative developer could really exploit.

Write your own.

http://www.devkitpro.org/category/devkitppc/

I'm only interested if it comes with a Wolfmother soundtrack.

Ahem.

http://www.youtube.com/watch?v=p2qlHoxPioM#t=0m22s http://www.youtube.com/watch?v=YKjV_bWnFCM#t=0m39s

They taught a computer to perceive the world? Most misleading title ever.

What? Maybe this comment was intended for a different story; the title seems fairly explanatory to me.

I think this sums it up pretty well ;-)

http://image.tutorvista.com/content/heat/water-cycle.jpeg

Reply on news.ycombinator.com