I doubt there is any kind of actual contract in play, so either party can choose cancel the relationship at any time. I would just send a written notice in addition to making sure they've heard about the cancellation on the phone (and recorded it). As long as they've been reasonably notified, you're not buying their services anymore.
Big business (and especially the collections agencies they usually send this kind of supposed "debt" to) love to say a lot of things, but that doesn't make it legal.
So they claim some account is "in collections", implying you have some sort of debt to them. If they say that publicly[1] and that hurts some future opportunity due to the "bad reputation", then a libel (or slander, as appropriate) lawsuit should be filed. While each case would be different[2], you create a lot of the mess by acknowledging their incorrect claims.
Business walk away from stuff all the time, and so can you. If any restrictions were desired, they should have been written into a proper contract beforehand.
incidentally; this is also why "identify theft" is a stupid term - nobody stole your identity, which is immutable. What someone did was defraud a bank to get money. You were not a party to that transaction (or crime). The fact that banks wan to be lazy ad not do proper background checks on people they loan money to does not give them the right to recover that money from a 3rd party, not does it put any amount of fault on that 3rd part). Calling such a situation "identify theft" instead of "lazy bank loses money and blame it on an innocent 3rd part" is a modern version of "they were asking for it" style victim blaming.
[1] I include Experian/Equifax/TransUnion/etc in this - despite. Saying something incorrect - with the purpose of advising another business that that you are probably an expensive risk - is the very definition of libel.
[2] As always, check local laws and ask a lawyer