The Mecca for cave diving is in the Yucatán and surrounding areas and the caves there are exceptionally warm. The surface water can be chilly, coming in around 21c at one of the many cenotes (sinkholes, which are the entry points into the system) as rainwater fills them, but the further into the cave you penetrate you will eventually cross the halocline, at which point deeper = warmer ocean water. It’s quite unintuitive but delightful to warm up after a 2+ hour dive. It’s common for divers to go deeper, pull their wetsuit open a little to fill it with warmer water (24c+) and then rise up back to the planned depth.
Cold cave diving is a very different experience and is usually found in Florida and Europe. Don’t recommend.