For Japan, shrinking population will mean an aging population with no one to care for them.
Hopefully their population decline doesn't hit them too hard. But that, and their lack natural of resources, could lead to bad things. Fascists and their ilk thrive in such situations.
[0] https://en.wikipedia.org/wiki/Slavery_in_Japan#World_War_II