You can build housing that doesn't get mouldy and is well insulated by having proper insulation and a modern ERV. It seems like Japan's policy of 'disposable housing' doesn't stand up to expensive electricity.
During the rainy season, humidity in (mainland and southern) japan gets above 95%, for 6-7 weeks.
And then of course, there's the local building habits, like the cardboard houses of the US.