"As the video shows, in our OpenGL ES testing, the 3GS is generally close to four times faster than the 3G. Results will vary depending on the application but this is remarkable to say the least."
It would have been cool if they had tested some real-world apps from the App Store. Not sure how they would be benchmarked, though -- load time?
Load time wouldn't exactly illustrate the improved 3D capabilities. I guess you'd need special builds of games so they show the frame rate (unless some of them already provide it) and ideally have time demos which don't require user input and instead use pre-recorded inputs and random number seeds.
You can use instruments to find out the number of fps. Very useful to optimize uitableviews.
On a tangent, anyone come up with any reasonable solution to simulating the iPhone's hardware environment from XCode (it so needs a slow-down plugin). It's nice to run stuff on my CoreDuo but painful to see how slow it runs when it's actually on the iPhone.
The only way to get accurate performance metrics would be to emulate the hardware at the silicon level. On the one hand, this is extremely difficult, on the other hand, it's also very slow. You're pretty much just going to have to suck it up and test on real hardware. Compared to certain game consoles, the iPhone tools are pretty good.
No. You need to test on the iPhone. The iPhone is not just a slower version of your Mac. It has totally different performance characteristics. And they have a different feature set even, like other than missing apps on the simulator, certain animations are missing in the simulator.
There was a vid that went through on HN the other day (sorry no link) that showed side-by-side app start times for one Iphone app. And the timing was phenomenally shorter on the 3GS.