Not diminishing the degree to which it can hurt your body long term - it absolutely can. That being said, most people that do that kind of work as a career start it as a job when they're 16-18 years old, and are typically active (sports, etc.) for many many years before that.

You're starting it at best in your mid-to-late twenties? And even if you played sports in HS and go to the gym regularly, your body just isn't used to it and it's past the point where it can quickly adapt and more importantly get used to the stimulus. You're likely much more prone to injury than someone fifteen years older than you whose been doing that work since they were 17 (I'm sure I am too, it would destroy and I am very active for a SWE).