1. Because about 90% of web users were on awful Microsoft browsers which was extremely slow at processing JavaScript, and made a very bad job of rendering CSS. Now, 80-90% of users are on a modern browser.
2. Google's methods of indexing and scoring content changed - it now parses JavaScript, gives value to speedy pages, is good at detecting keyword stuffing, etc.
3. Always-on internet is now pretty much a given, even on phones.
4. There are more developers, trying more things, sharing more experience.
5. Tools. In 2006, we had Drupal, Wordpress, Plone, and a few other odds and ends. These days, we have Varnish, nginx, Jekyll, CloudFront, MongoDB, etc. There are a lot more options, without having to spend a fortune or build everything yourself.