Unplugging it and plugging back in will sometimes fix the problem. Looked into it and it turns out the power relay is getting stuck in the on position. I have to open the dehumidifier completely to get access to the relay and then tap on it with something hard. This has worked to unstick it, at least so far.
I think maybe this relay isn't up to its task. I was curious about replacing it with something similar but couldn't find anything remotely as small for switching 120 VAC mains power.
For reference, my dehumidifer is a Toshiba TDDP7011ES2. The relay in question is a Sanyou SFK-112DMP.
I dunno, maybe this will help someone out there figure out what's going on.