The inverse is not true. See the early reviews of the Kindle Fire HD, where the device is described as sluggish and laggy despite having better specs than the Nexus 7.
So the fact that something has faster hardware does not mean the one has a UX advantage over the other.
Also, bear in mind that the mini doesn't have to push around 4X pixels like the Retina iPads, so it can get by with less RAM and processor.
I, personally, would buy the Kindle Fire. Most people, however, will buy the iPad Mini. Bigger price and all.
Curiously enough, I think the iPad 2 has just become the most compelling tablet in the market in terms of price/specs/performance. I'm wondering if iPad 2 will eat into sells of the new iPads? And would Apple care?
Anyway, interesting times.