Don't do this... please don't do this. Just set all your taps (internal and external) to drip slowly. That's all you need to do. Shutting off the main could actually make it worse because it creates an area without expansion.
> So what would I do to "winterize"?
Consider putting in more insulation in your attic, R30 should do it. It will help with energy bills in the summer too. Fixing walls is nice but not worth it unless you're doing a whole house reno.
Also next time you get your AC replaced, consider swapping it for a unit that also functions as a heat pump. Get one rated to 5-10F, that means it will be within rated efficiency range until that point. It will still function and operate below that but will be more like the electric heat you have now below that. But still generally more efficient overall. That reduces load, less load overall means less chance of things going out. It may seem small but those two things can have an effect.