short version:
Don't upgrade just for speed. i don't see a difference, have never heard of a difference in performance.
long version:
I use both. In my experience there is no difference in performance. Very occasionally a bug that hits one will not hit the other (but not always). Also, they get features at different rates. The consumer Google applications get new features and makeovers faster, and then it gets rolled into Google Apps.
It sounds like you looked at the feature matrix and know the main differences. Basically Google Apps gets you a throat to choke (there is support), and access to more API's to do stuff programatically. If you're just one account, a lot of that value can't be realized.