Hiring managers care because it's a big hit to your deliverables every time someone leaves and you have hire someone else, bring them up to speed, and try to get them back up to the same level of knowledge as the person who left.
If you have a team of 6 and everyone leaves every 12 months, that means you're hiring and re-training a new person every other month. And every other month, 17% of your team's accumulated knowledge/experience walks out the door. And every year, you're basically starting over with a new team.
It sucks, and that's why hiring managers give preference to people who aren't serial job hoppers.
It's actually becoming fairly common for companies to give back-weighted compensation or hiring bonuses that must be paid back if the employee leaves before two years, especially for job hoppers.