This was the case until the mid- 2000s (to the point where if you didn't take money out before New Years, you might find yourself penniless for a few days).

Over the last few years 24/7, 365 days/yr ATMs are have become much more common. As have ATMs that accept foreign banking cards. As has a more generalized acceptance of CCs.