But there is an important distinction between global and local symmetries. Global symmetries, when you apply the same change to the field at every point, are physically meaningful, not so local symmetries, when you apply [potentially] different changes at different points.
For example the laws of physics are time translation invariant, i.e. it does not matter what point in time you call t equals zero which essentially means that the equations do not contain time but only time differences so that you can add the same constant to all your times and nothing changes as the constant cancels out when calculating a difference between two times.
Its the same with voltage, where you put your reference potential does not matter but this is again a global symmetry and you have to use the same reference potential everywhere, you will obviously get nonsense if you use different reference potentials at different points.
Local symmetries on the other hand are kind of defects in the mathematics of physical theories, they are the expression of redundancies in the mathematical description. Say you want to describe the orientation - but not the strength - of the magnetic field on earth and for simplicity lets assume the earth is flat and the magnetic field parallel to the surface, then you could do this by associating a two dimensional vector with each point on earth that describes the tip of a compass needle placed at that point.
But there is a problem, there are longer and shorter compass needles but that is irrelevant for the orientation of the magnetic field, the length of the vector does not actually matter. You could multiply this vector field with a different constant at every point, i.e. independently change the length of all the compass needles and you would still describe the same magnetic field orientation across earth. What you do to fix this is to declare that all fields are physically equivalent if they only differ by a constant factor - which may depend on the point - at each point.
The other solution is to use a better mathematical representation without the redundancy, instead of compass needle tip vectors you use the bearing angle and take the compass needle length out of the equation to begin with. Problem solved. Now you can no longer change the field value, i.e. the angle, at every point independently and still describe the same physical situation. Also note that there is still a global symmetry, you can still change all the angles by the same constant, you are free to pick which direction you label with angle zero, which is again physically meaningful and expresses that space is isotropic, i.e. there is no preferred direction and the equation therefore do not depend on the direction but only on the angle between directions.
And you can do the same thing with the known physical laws, you can for example get rid of the U(1) symmetry - which should better be called a redundancy - in quantum electro dynamics. The price you have to pay is that the resulting equations lack some other properties often considered desirable, for example they are no longer obviously local.