Prior to this, I had already refunded the full deposit that was paid to me, for not completing the project.
It is an insane amount of money, and I do not know what to do now.
Do not say one more word on the subject. Not in writing, not in public, not even to your friends over beers, until you get a lawyer. Think of it like the Miranda rule -- now that a legal dispute has arisen, anything you say can and will be used by them against you.
What you shouldn't do now is panic. They've sent you a letter demanding a large sum of money? Big deal, anyone can send a letter. It's not unusual for people to try to bully others by having their lawyer draft a letter full of scary-sounding language that, when parsed by a lawyer, actually says nothing. These people hope you will read the letter, panic, and do something stupid like trying to settle with them before talking to a lawyer.
Don't give them what they want. Get a lawyer.
I suppose I should still go get a lawyer right?
It's possible your email won't have done much damage. Don't worry about it... just get a lawyer.
When you've found a lawyer, he/she will advise you on what to say and not say from that point forward. In the meantime, the safest course is to say nothing (or at least the absolute minimum you can) from now until then. If you've already sent an email you can't take it back, obviously, so when you get a lawyer be sure to give him/her a copy of the email (and any response you receive) so they can advise you on how it affects your position.
Good luck!
Ultimately this comes down to your contract with them. You did have them agree to a contract detailing everything right??
Although some will say a contract is worth as much as the paper it's written on (or how much one is willing to pay a prosecutor) it's still smart to outline expectations from both sides before beginning work.
I'm curious though -- define "insane amount of money".
Also, what went wrong? Unmet expectations? Better offer come along? Burn out?
As for what went wrong, it's a case of both having a better offer (work at a startup) and burn out from bad experiences with other clients.
This won't hold up in court and it will only get worse if you ignore it.
Good luck!
That said, I doubt it's as bad as he fears. He needs a lawyer.
Get an attorney, who'll probably advise you to seek an out of court settlement, if possible.