Chicken/Egg problem for sure. Niche effect that Amazon had trouble encouraging app makers to build for (lots of effort for no users), and since there were no blockbuster apps, users didn't buy it.
The intention was that the UI would reveal extra info (like meta-data about a photo, or calendar details) when you shifted your head POV. Like a gesture that you could do one-handed while riding the bus. (But not discoverable, and twitchy, and tail-wagging-the-dog.)
Given how coarse the tracking had to be so that the UI wasn't constantly shifting, it would have been far simpler and more energy efficient just to use the accelerometer instead of the fancy camera setup. And just use that for some nice parallax effects in the launcher; not as a core UI interaction.