I don't know the full context of what is going on legally, but I do know that being threatened is not the same thing as getting sued. And getting sued is not the same thing as them having a case. And them having a case is not the same thing as you not having a defense.

Lawyers exist for a reason. Don't let other lawyers bully you with threats. Get your own lawyer to look into things. Back down if it makes sense. Sometimes it does. But don't back down just because you get threatened.