Austin, TX area:
We used to live in 1800sqft house built in 1992 in the affordable neighborhood (so fiberglass bats in 4x2 walls). Gas heater, electric cooling. Gas water heater. Gas stove (we don't cook much, so this one probably doesn't have significant effect). We regularly got a bill over $250 in the summer for electricity + gas. Sometimes over $300. In the winter combined bill would be around $150. In summer second floor was never comfortable (part of it was poor circulation I am sure, but part of it was HVAC not cooling well enough)
We also just finished new 3100sqft house. The new place doesn't have natural gas, so we decided to go all-electric - water heater, HVAC, cooktop/ovens, and new addition - pool-heater. Our electric bill now fluctuates between $200 - $280 year around. All appliances, including water heater, are heat pumps. Extra bonus - garage, where the water heater is located, is cool during summer. Both water heater and HVAC has resistive heating elements for very cold days (we get enough of these to matter).
I am sure I can get average bill lower a bit if I tweak pool water and heat pumps schedule.
The house has open cell foam installed around the envelope (so the attic is alright to be in even during the hottest days) and we have forced ventilation (since the house is sealed essentially).
The water heater (80 gal) I bought myself and it was $900 on ebay. Similar product bought at Home Depot would cost probably $1,200 - $1,400. A bit more than natural gas heater, but not prohibitively so. HVAC installation (2.0 and 2.5 tons units) was under $20k for the whole house. Not sure how much traditional gas+electric non-heat-pump install would be, but I don't think it would be significantly cheaper.
So yeah, at least for southern places, heat-pumps appears to make a big difference.