I started with my old Vermont home with minimal insulation, single pane metal frame windows that would ice up, and drafts everywhere. After our first winter, we had the local efficiency program recommend a contractor who came out and measured air exchange and identified the order of work. New windows, sealing attic, re-insulating attic. We got a considerable tax credit for the work, and the comfort improvement was astounding.
Just this last summer we made the jump to mini-split heat pumps. They are working well, but I doubt that would be the case without the air sealing. We keep them set at their lowest, 61F, and supplement with a wood stove in the living room, maybe burning 2 cords a year.
As an aside, this is our first winter, and our bill for December was 300 dollars, at $0.18/kWh. That's about 25% lower than our estimated oil charges last year for the same period. However, this year oil is ~$4.6, almost 50% higher, so that hedge against fossil fuel price is really paying off. Add in the AC comfort that is largely offset by our solar array, and they are a worthy investment (oh, and 0% financing).
None of this would be working without that critical investment in air sealing and insulation.