Instead of charging per KW/hr, charge based on the actual costs to the electric provider. A fixed monthly cost representing the cost of upkeep and being connected to the grid, and a variable cost portion that actually reflects usage.
Yes, this doesn't encourage reducing electricity usage, which isn't as great for the environment.
But if solar is getting large enough that this is becoming a problem for the utilities, then the answer is to fix the pricing issue - not to stop putting up solar.