Why do you think its in the margin of error? I never mentioned the error bars on the statistic - We did an extensive AB test and it was a 3% stat-sig improvement
Unless your initial page load is taking like 10 seconds, a human wouldn't even notice. And if it is taking 10 seconds, well, then you've got better things to fix.
That's not an actual improvement. That's just noise.
Imagine you lower the grip on everyone’s tires by 3%. Most people wouldn’t notice, but for a few people it will prevent them from being T-boned when they wait too long to brake for a red light. That improvement would be measurable if your data was good enough. I’m not saying optimizing your website is important like preventing car crashes, but it’s an illustration of how a small change can lead to a measurable improvement.