Depends on how quickly the captain was able to disable the CVR.
However if that theory is indeed true, it's clear that he wanted to disappear without a trace. In that case it would have made sense to keep it running especially because it only keeps the last 2 hours.
But yeah if it had been longer he would have turned it off in that scenario. It would be best if the CVR had a backup battery (and internal protection is that shorting out). In fact I remember reading in several admiral Cloudberg articles that the CVR and CDR data was incomplete due to bus power loss during accidents.