Carbs soak up water, so it's natural that you would gain weight when adding carbs and lose weight when you remove them. That's why doctors and nutritionists say not to worry about your weight, but worry about your waist. Weight can be misleading.
I would certainly worry about 20 lbs gain, carbs or no carbs.
After switching to keto, glycogen is used up gradually, due to no carbs entering the body. And the water is also lost. Therefore, the "signing bonus" of the keto diet - a significant weight loss in the beginning, but due to water lost, not fat burned.
BUT. That only accounts for about 4 kilos, maybe just a bit more. You can't really blame water for a gain of 20 pounds.
Am I missing something?