The site is hosted on S3 and cloudfront right now. The poor performance might be because the page transaction and animations.
My initial vision is display the album similar to a photo book, now seems nature scrolling might be more acceptable.
A good example would be a Facebook photo album. If I click the left arrow as fast as I can, the page keeps flipping instantly. Instead of seeing each picture, I see a small loading icon and a box where the picture will be. When I finally stop mashing the arrow, the picture I'm settled on will load.
I hope you don't find my comments overly critical. It's a beautiful design, just trying to be constructive.