Yes! No! (as some others commented - not too hard).
You can build an amazing body (not hitting on you dude!) with Yoga. Which starts easy, then can turn hard, and 4-5 years later you got a body of iron, flexibility to reach the highest shelves, and stamina to run for hours.
Anyway, depending what you like/enjoy doing, build some muscle, build some stamina, drink less (alcohol). Be healthy(-ier)