Yep. It's usually the wrong tree species, wrong placement, or poor maintenance that leads to problems. I had 100'+ pine trees in my hard. Beetles were somewhat of a problem but the woodpeckers were all over them and systemics helped.

Grew up in an house with 60' Australian willows that became too fragile and messy and were too close to a structure. Also, never get lemon trees because thorns from hell.