Some numbers: Swedish electricity production in 2023 was 163 TWh. It's for all practical purposes 100% renewable or nuclear, with a mix of approximately 40/30/20/10 hydro/nuclear/wind/other, where "other" contains 2% solar and the rest is pretty much just electricity byproducts from district heating.
Also in 2023, district heating delivered a total of 53 TWh to customers. About two thirds of this came from waste and biofuels, specifically 47% renewable biofuels (mostly woodchips, bark and other forestry industry byproducts) and 20% waste. The remaining third came from an eclectic mix of inputs such as industrial waste heat (7,5%), heat pumps (5% net, that is after the electricity consumption has been subtracted), exhaust gas condensation (11%), biogas from landfills and compost (2%), fossil fuels (2,5%) and various other stuff. Turns out my info about peat was kinda out of date; it's been almost entirely phased out now.
Average CO2 emissions for the electricity generation was 41g CO2 equivalents per kWh, and for district heating about 55 g/kWh. Countries with less emissions than this do exist, for example Norway and Iceland are down around 30 g/kWh, but this is already better than almost every other country on the planet. US electricity generation for comparison averaged 369 g/kWh in 2023.
So: replacing all of the district heating with heat pumps would be an insanely huge project requiring a massive expansion of electricity generation, and it would have very low return on investment when it comes to reducing CO2 emissions. There is also the problem that heating energy demands are extremely uneven and unpredictable, both over the year and between years. You need a significant amount of extra capacity for very cold days, and even averaged over the year the difference between an average winter and a cold one can on the order of 20%. Just burning stuff works great as a solution for this, it's easy to have capacity on demand. Air-source heat pumps also become much less efficient on very cold days when you need them the most, which is why they're not that popular for heating here (people prefer ground source heat pumps, but those have limitations in densely populated areas because the ground doesn't contain infinite energy).
Also, a side note: the heat generation in district heating plants is extremely efficient compared to electricity generation using combustible fuels. The average efficiency of the heat generation in these plants in 2023 was 96%, and that's before exhaust gas condensation is taken into account.
In other words: there's a reason we are doing things this way.