Any remote control's microcontroller would be in deep sleep most of the time, and computationally expensive key exchange would be rare.

I have a few of these ESP-8266 remotes, and battery life is fine. https://github.com/mcer12/Hugo-ESP8266