Ask HN: Why are patent holders allowed to sue years after their patent was infringed upon?

It's obvious that patent holders/trolls wait until their infringed-upon patent becomes profitable before they decide to sue the companies that made it profitable. Why is this allowed to happen? Shouldn't the patent holder be responsible for defending their patent as soon as they know that it is being violated? It seems we would have less problems with patent-trolling if the prize at the end of lawsuit wasn't so big.