Athletes do not make it to the Olympics through "brute force." Exercise is the least part of Olympic training. Olympic-caliber athletes spend years practicing their specific discipline(s), developing everything from the mechanics of their movement to the interplay of their body with the environment.
Cross-country skiing isn't just something you can "pick up" a few months before the Games and hope to have a shot. It is a grueling physically demanding sport that requires tremendous endurance, technical, and mental preparation.
Also--simply being a citizen of a low-population country is not sufficient to qualify for the Olympics (winter or summer). The Olympics are a competition for the best athletes in the sport--novelty acts don't just get to waltz in. Prior to each Olympics, each sport sets forth the qualifying standards (time, place in qualifying races, etc.). Even the Jamaican bobsled team had to qualify for the Olympics.