Pinky: Why? What are we going to do tomorrow night?
Brain: The same thing we do every night, Pinky - try to take over the world!
Clearly a mutual ancestor of chimps, gorillas and humans did develop human-like intelligence. I'm not sure it's meaningful to ask why, say, chimps didn't develop in that way since they ended up being a subgroup subject to different evolutionary pressure.
Evolution isn't magic, it's just in an inherent process of an adaptive system. Like I wrote above, I think of it as a huge, loosely coupled hill-climbing computation, but the different nodes run at different rates and interact. You could also think of it as a neural net without a directed training system.
I think your second question is very interesting and I have wondered about it too. Ignoring the case of the neanderthalers for a moment, it's likely there were lots of variants who just suffered some environmental issue (e.g. a drought where they lived, or an invasion from some carnivore) that impacted a population too small to survive the threat. (and for purpose of discussion we presume this group were legitimately speciated from us). This happens to animals all the time. So we look back with survivorship bias.
The neanderthalers are more complex since they did survive until very recently and may not have been a completely different species. My guess is the threat they faced was homo sapiens sapiens. Is that an inherent property of humans? Seems like it :-(.
BTW by "human-like intelligence" I assume you mean "able to make complex tools that multiply its own functions in a non-linear fashion, support a complex society, and develop the capability to live apart from the earth". Since a cockroach civilization would have different objectives and metrics from ours their behavior wouldn't look anything like ours either.