I don’t follow. Is the implication here that users can identify an experience difference cause by a 30KB over-the-wire asset, or more, a 90KB in memory one? I know I certainly wouldn’t claim to have that ability myself even on low quality networks.
If you're a user on a spotty mobile connection in a rural location on a low-end device ... both.
In that case, most of the web is unusable nowadays, and jQuery is not your main problem. Maybe your argument is valid in less-developed countries where more people have spotty connections and more websites have adapted to that (but I wouldn't know, sorry).