Using WebPagetest, you can test your page with real devices to get an idea of how it will load for users who are not on overpowered desktop devices hooked up to the Internet over a fibre connection. If you look at the "Filmstrip view" for futureclaw[0], you don't even see any text until ~4.7 seconds in, and the page doesn't look finished until well after 13 seconds because that image takes so long to come in. Hacker News, on the other hand, has content visible at 2.3 seconds[1].
So yes, back-end times are important. But it's a small slice of the overall load time. Just keep in mind that the proportion of people on mobile connections is actually going _up_ in the developed world, and the average speed of mobile connections is not improving. It's really important that we (web developers) recognise this and build sites that can be used by everyone, no matter the speed of their device or connection.
[0] https://www.webpagetest.org/result/161224_TS_264/ [1] https://www.webpagetest.org/result/161224_H9_26Z/