This looks great! A nice feature to add would be to map the mouse wheel scroll with javascript to scroll horizontally, so you could also mouse wheel scroll through the site. It's a simple javascript snippet to add, if I remember correctly.
Great performance too, in Firefox especially (Chrome seems slightly choppier). Browsers put a lot of effort into optimizing scrolling these days (buffer rotation in desktop Gecko, tiles on mobile Safari/Chrome/Firefox), so structuring your interactions around it is an easy way to get excellent performance out of HTML.
Great idea, cool website, good luck with the project! Looking forward to being able to use my N64 again without expletives due to the brittle controllers!