Really the TTL (Time To Live) should allow fairly quick DNS propagation. The problem with TTL is that you need know your change is coming in advance.
Eg: If your domain's TTL is currently 24 hours, you need to change it to something much shorter (say 5 minutes) at least 24 hours before the DNS change happens, as this will mean that every DNS server will know to update its record for your domain every 5 minutes before you actually make a DNS change.
The other problem is to 'improve' response times, some DNS servers (particularly ISPs) don't honour the TTL directive.
Of course it'd be nice to push changes quickly, but I doubt there is any feasible way to coordinate that on the internet. The next best thing you can do is to lower your TTL in advance.
For example, if I said end users could run their own caches, use their own custom zones, or custom /etc/hosts files for sites they frequent, I'll bet the idea would quickly be shot down by the defenders of the status quo.
Yet it solves the "so long" problem. They can honour the suggested TTL's the authoritative DNS servers provide, or set their cache's min/max TTL's as they wish. Long or short. End user decides. Not all sites, including CDN's, keep changing their IP every few minutes. Once the user knows the closest IP for getting what she wants, no need for the DNS. Things become very fast when you don't have to constantly look up the same names, day after day.
But the real question is why are you asking why it takes "so long"? What is the real problem you are trying to solve? For example, are you needing to keep switching IP's, add CNAMES, or create more indirection? If so, why is that?