The majority of foreign-born people in Sweden did not arrive as refugees. About half were born in Europe, with Finland - which is very similar in culture and also has conscription (as well as a much larger army) - being the most common country.

Source: http://www.statistikdatabasen.scb.se/pxweb/sv/ssd/START__BE_... (official gov't statistics)