Having coworkers just showing up and leaving at any random time during the day I think messes with morale/vibe of the office. And as another commentor pointed out, communication. I like to know the general time-range I can find an employee to speak to. I'm old-school and still appreciate face-to-face time over online tools. If the expectation is set at the beginning that "Jane is a telecommuter Mon,Wed,Fri". Ok, that's fine. I don't like the idea of "Jane just shows up whenever she wants to, but she's getting her JIRA tickets done within the sprints and shows up to meetings".
Basically, while schedules definitely don't need to be rigid... they also shouldn't be fully impromptu either. If you're not telecommuting, I believe it is valuable to be physically around during office hours.