I'm no expert on UK energy pricing, but the main difference between a boiler and heat pump is that heatpump can be, eg "500% efficient" - a COP of 5. So even if a KWH of energy is 2x the price for electricity a heatpump often comes out ahead.
And OP is right. When I've looked into it you aren't going to save any money going to a heat pump; in fact it will likely increase costs slightly.
So it really only makes sense to do when you need a new boiler. If you don't, it makes way more sense to get solar because that's also good for the environment but actually saves you money.