Anandtech had a good overview page of what 64-bit (and more generally, Armv8) does for performance:

http://anandtech.com/show/7335/the-iphone-5s-review/4

Basically, the new arch increased registers, increased register widths, and cleaned up some architectural warts in Arm that hurt performance.

The "4GB memory" issue is a red herring -- 64-bit is about making all the paths through the chip wider to process more per cycle.