How is physical strength unimportant? Stronger people are harder to kill (h/t Rippetoe) and improved strength also helps to avoid carpal tunnel syndrome and neck/back pain for those of us at a computer all day.
Most predators will kill us even if we're ridiculously ripped, so that's not important either. In most cases, intelligence seems a lot more useful for avoiding or navigating these situations.
There's also a lot to be said for having moderate strength in a wide range of motion. That can matter a lot in a much broader variety of situations where physical strength matters, and it's far more realistic to attain and sustain.
Avoiding carpal tunnel doesn't require special levels of strength so much as normal fitness. It requires regular movement and preventing weaknesses. I'm not saying we should all vegetate at a computer, I'm saying that the evidence seems good that being 5% larger/stronger (or whatever the figure was) is not particularly helpful, and instead potentially harmful.
For example, regular yoga will probably resolve most (not all) people's mobility and repetitive stress issues. They will not need to be large or strong for this to work.
If you look at the top X causes of death in humans, very very few of them could have been avoided "if only the deceased were stronger."
It's also pure speculation. I read about this stuff for fun and I know just about nothing about anything. Maybe being larger really is a net positive and I just need to be pointed to the right data.
If someone is relatively weak with below average lean mass, lower than average markers of muscular strength, etc. it seems like the consensus is that they are statistically likely to die or get diseases earlier in life. On the other hand, being on the other side of that seems to yield diminishing returns. It’s great to be fit, but being abnormally strong on the other hand doesn’t appear to increase your lifespan proportionally to your strength.