And much of our power comes from solar (which is maximally available at the time cooling is maximally needed) and nuclear, while serious heating is often literally burning fossil fuels in your basement.
What does it take to spend a vacation day outside working in your garden? At 30F that's just a thick jacket away, what kinda clothing can make your body work at 120F and sun?
I will say there likely isn't a ton of good hard data on this for another reason, and that is that humidity plays a role, both in what temperatures are comfortable and maybe in some forms of temperature management.
The big difference is insulation. On the average, houses that need heating are better insulated than houses that need air conditioning.
Geothermal heat pumps and good insulation solve both.