But you can simply bicycle out (as the stream of Syrian refugees entering Norway from Russia illustrates). Ergo, you cannot simply walk out of Russia (to Norway) either.
This was happening at the Finnish border, too, but just recently Finnish authorities banned crossing it with a bicycle, officially for safety reasons.