Removed my water heater and HVAC, for two 500 feet deep holes in the ground. Electricity bill is low, and gas bill is gone. You can receive tax breaks from federal, state, and local utility companies (in the US) too.
While I do recommend it, you may need to wait for an available drill if the terrain under your property is difficult to dig through.