To not load openstreetmap's servers, you could pull the tiles to your server and host them. Then only grab a tile from OSM when you are missing one.

This would also let you set the grayscale on the tiles using imagemagick convert.

Neat project and thanks for the write up.