I ran some experiments on this when I first joined Google. With JQuery loaded off ajax.googleapis.com, running my tests on google.com, cache hit rates were roughly 87%. This is with a CDN that caches everything indefinitely and the most visited site on the web at the time.
There's an extremely long tail of people who browse the web only occasionally, or have just one favorite site they go to, or who are running on devices with small caches, or who just cleared their cache because they wanted to look at porn. The hit rates are probably even worse now with the shift to mobile, because mobile devices have tiny caches. In recent prototyping experience using Chrome ADB to check on network traffic, files are gone from cache after only a half dozen or so other pageviews.
You can't assume "caching will take care of it" if the site downloads several hundred K of JS. Caching is effective at making an already-lean site even faster, and you get even more bang for your buck because more pages can fit in cache when the pages are small.
Anyway, I was thinking mostly about developer experience when I wrote that comment (this thread is about MVPs), and also mostly about mobile development, where the bulk of my time is spent lately. JQuery is a noticeable drag when loading a page over a cell network, even if you just want to try out some ideas. It's not just network latency, either; on mobile devices, you can burn significant time (and battery) just parsing and executing all that JS.