Many/most hot water bottles have a knitted fabric cover. When the heat runs out (which takes many hours), you are not in direct contact with the rubber/plastic body of the water bottle, so there is no sensation of cold or clamminess.
An alternative would be something like a bag of rice or cherry stones. You can microwave them and they won't feel cold when they lose their heat.
Hm, the ones I am using usually are still warmer than body warmth on the next morning... YMMV
The coldest they become is body temperature under the blanket. It would only have a chance to become colder than its surroundings if the water had a way to evaporate, which it doesn't.