Great work! What about rendering it to SVG instead of canvas, with a buffer area around and let the scroll take care of itself?
Personally I found canvas easier to work with, scroll performance was better, less browser bugs, drawing was cheaper and scales for large content (using virtualisation or multiple canvas layers)