How do they get the efficiency number above 100? Are heat pumps that efficient at cooling as well?
A heat pump moves heat from outdoors to indoors. The outdoor heat lost is ignored in the efficiency calculation (as it should be), and thus the efficiency question is how many watt hours of electricity does it take to move a watt hour of heat (joules) from outside to inside. Depending on the temperatures involved, it's usually a multiple, for example, 1 wH of electricity moves 4 wH equivalent of heat indoors, ie: 400% efficient.
If you consider the whole system, much of the electricity for a heat pump may come from burning natural gas elsewhere, which involves losses along the way. This is still usually better than 100% overall efficiency.
Direct natural gas heating is less than 100% efficient too. Most furnace systems are about 20% loss in exhaust, though they can be much better. This is still usually better than burning the gas at a plant, converting to electricity and then heating with resistive electrical in your home.
And comparing AFUE efficiency to HSPF is not really a thing, you can not look at a gas furance that is 95% AFUE, and a Heat Pump that is "400%" claim that the gas furnace is must less "efficient" that is not how any of those works. To be clear a heat pump at temps above 25 degrees Fahrenheit is more efficient but comparing the metrics in this manner is a bit misleading
Further heat pumps are good to 25 to 30 degrees Fahrenheit, under that a gas furnace is normally used to supplement a heat pump in colder climates. Most heat pump manufacturers do not recommend operating a heat pump in below freezing conditions
Just wanted to chime in that they do make a lot of low-temp heat pumps, going down to -30f now. The efficiency drops dramatically at temps that low, the best site to look at cold-weather models is https://ashp.neep.org/#!/product_list/
If you look though, the "COP @ 5 degrees F" for a lot of models is above 2.0, which from my understanding means you'll be using 1/2 the electricity someone with baseboard (resistance) heat would. And above those temps the COP gets much higher.
They can use 100 watts of electric energy to move 400 watts of heat from outside to inside.
The downside is that the colder it is outside the less efficient they get (since there is less heat outside) so it only works in relatively temperate climates.