My assumption is that intentionally copying IP onto a personal device and removing that device from the office -- regardless of any actual intent to use that data -- is still theft. Which would make the former employer's claims truthful.
It's super unclear to me how you would accidentally retain a copy digital documents...?
Like I said, it's hard to imagine this scenario actually happening. But for good measure:
Lesson #3: Leave work at work and startup at home.