Were the old Rayovac batteries alkaline or zinc-carbon? Zinc-carbon batteries use a "dry cell" chemistry that will never leak, have a low self-discharge rate, and are very cheap to make and buy. You typically find them marketed as "heavy duty." The trade-off with those are that they have a lower energy capacity compared to alkaline.

I use NiMH in almost everything but for things like TV remotes and clocks that sip power AND you can't be arsed to change them all the time, they work great. I buy a pack of 4 at the dollar store.