That font is almost impossible to read.
In my case that was much uglier, and I had to fix it to read the page.
While this is a reasonable explanation, I see many cases where system fonts set wrong and as a result an unstyled page look terrible.
I saw the problem with Firefox 10 on CentOS. My default font was specifically configured to Bitstream Vera Serif.
Setting the default font in the Preferences seems to only change it for the encoding in the active tab (unless you go into the Advanced dialog). For several Asian languages it wanted to use "serif", which appears to be the 20-year-old X11 default version.
The page is hard to read on Mac using unmodified Firefox.