If you'd rather not wire it up yourself there are also ESP32 dev boards with built-in battery management functionality, such as the LoLin32 Lite and Sparkfun ESP32 Thing. I haven't had much luck with the former (possibly due to its lack of RF shielding) but the latter seems to be pretty solid. I think Adafruit sells similar boards as well.
[1] https://hackaday.com/2022/10/10/lithium-ion-battery-circuitr...
ESPHome also has deep sleep support - so for some use cases you can just wake up every x minutes/hours, connect to wifi, do thing, back to sleep for x minutes. In deep sleep a decent ESP32 board (firebeetle or tinypico) will last for months on a small lithium cell. For a quick sensor, the whole wake up/read sensor/update HA/sleep again takes a second or so depending on wifi configuration.
Useful for something on a schedule like sprinklers or slow sensors (soil humidity or whatever).
You can also wake based on interrupts, which is good for stuff where you are using a low power external sensor that does interrupts (wake ESP up if humidity gets to x) or a GPIO switch (magnetic entry/float switch/etc etc).
Firebeetles and tinypicos both have cell connectors and onboard charging directly for lithium pouch cells. You could also get a cheapo solar power bank, although you'll want to do some research to make sure the relatively light load of an ESP32 will keep it powered on.
They also have a wide variety of sensors that connect with a ribbon cable (they call it uEXT) with no soldering required. Many of the sensors are supported by ESPHome.
Featured yesterday on HN for being acquired by Espressif.
For indoor use, I made this to power ESPhome devices from a cheap apple USB adapter: https://www.printables.com/model/703859-esp32-enclosure-with...