Yeah, if you own a copyright or trademark and do not defend it, and can be shown to have known about it, then it becomes almost impossible to defend later on in court. Kind of like a muscle, if you don't use it, you lose it.