You should have ensured that they had possession of a reliable backup/recovery method -- or, at least, clarified such a need via your exit interview.
For most company the exit interview is done to tick a box. The feedback given will not be listened to at all. I can absolutely understand why he didn't feel like it was his job to setup and entire source control system on his way out the door.
The company did not mistreat the employee, just that it undervalued the employee.
I think the situation is analogous to the company giving the employee a big unwarranted pay raise, and then the employee lazily not doing more work, then the company firing the employee, but then arguing that at a minimum, the company has a reasonable professional courtesy to give the employee hours for their vacation days, even though it isn't specified within the contract.