Here in Texas, electricity is only about $0.11/KWH so heat pumps work out like this:
Heat Pump 5.0 COP : $6.50 per MMBtu Natural Gas : $12 - $15 per MMBtu Propane at $3.00 / gallon : $32 per MMBtu
The unfortunate characteristic of heat pumps is that they become less efficient the more you need them, i.e. cold weather, but if weather is generally above freezing, heat pumps are close to their spec range that is based on 47F outdoor temperature.
A buried heat exchanger or a water well makes it a weather-proof geothermal system- this seems like the ideal; if you also need air conditioning it will make your cooling more efficient too.