Regarding ugliness - I started on a stylesheet, and then just decided that wasn't the point. I just wanted something quick to demonstrate this use of KBFS, and I'm not expecting (or desirous of) people to use it as a genuine service.
I've been working a bit on a blog for my own ramblings (some here may have tried the naked domain and noticed it 503s) - I'm now interested in doing that as the demo here but fixed to my account. (And uh.. with a stylesheet and stuff!)
As you noted, it would then seem sensible to do some caching - but I think this needs careful thought to not destroy the purpose and benefit of hosting posts on Keybase. I suppose it would have to display the cached copy, and then load from KBFS in the background anyway, to verify it (not only as untampered-with, but also as being still the latest version).