NTP and even SNTP are performance intensive on the server side and so are generally configured to run at a low interval (either a fixed interval or, on Linux, an interval that becomes longer as local clock skew becomes known). This means that, at any random point, there's probably some reasonable amount of local clock skew that has accumulated since the last synchronization despite attempts to compensate by estimating the skew (essentially local skew is not constant, so estimating and offsetting only helps so much).
There is a tradeoff between highly accurate timekeeping vs. the cost of providing the service, and since the vast majority of users will never notice an inaccuracy of even a few seconds, OS vendors default in the direction of lower cost to maintain their NTP infrastructure.
There can be a bit of a tragedy of the commons situation since equipment which relies on the volunteer-operated pool.ntp.org is also somewhat more likely to have their NTP clients configured in an aggressive way that drives up load on that public resource - since the people configuring the client aren't responsible for operating the infrastructure.