I have a 3000 sqft house in MA with a heat pump as my primary heat, and an electric strip as backup. The heat pump goes into the ductwork for most of the house. I generally haven't had problems. (It did have to be adjusted because it was the first time the installer put one in.) I have solar and usually get 1-2 large bills in January and February, and then March and April are very cheap, if I get a bill at all. My "large" January and February are comparable to what I'd spend on conventional heat.
Last weekend we had a "once in a century" cold snap down to -4. Most of the house was at temp, but one room fell to the high 50s. From what I've heard, plenty of people with conventional heat had far worse issues than a chilly room.
When I built my house, I called around and it didn't sound like geothermal works very well where I live. I suspect the reason why I was talked out of it is because my yard is very rocky.
With an air-to-air heat pump, you need a backup system for when it gets too cold out for the heat pump to pull enough heat out of the air, and for power outages. With a ground loop heat pump, you only need a backup system for when the power goes out because the ground water will never be too cold for the heat pump to pull heat out of.
Heat pumps are now the most premium option on units offered by the well known names in HVAC- Trane, Carrier, etc.
In terms of suitability for your climate, they should work just fine for most people- they start to struggle if you are in an area when temps get down to about 10 degrees Farenheit, though they improve in this area each year. Even if you live in an area that cold, there are options to mitigate this- leaving your current furnace hooked up to be turned on during those cold periods, etc. What it comes down to is it just gets more expensive if you live in these colder areas, but over time the energy savings should give you a return on your investment in less than ten years.
My Air conditioner blew this fall, so I am actively in the process of getting bids for heat pumps.
If you're in 1, 2, or 3, heat pumps are definitely the right option. In zone 4 or 5, newer cold temperature heat pumps are a good idea. In 6, 7, or 8 it's less of a sure thing, and you will likely want a backup source for the colder days.