My challenge to the web performance community | Hacker News Reader

My challenge to the web performance community

philipwalton.com

61 pointsasicsp54 comments

If you're going to talk about the performance of a production site, use real-user data.

To do that I'd have to implement features that track users, or at least report back to a server with data I could use to track users, and I don't want to do that. As unfortunate as lab-data-only tests are, I'd rather accept those limitations than gather data I don't have any genuine need for. Heck, I'd rather deliver a slightly slower website than do that.

Some websites probably do have a need for super accurate perf metrics, and the devs who build those sites will have to wrestle with the question of what data get to from users for themselves, but I believe we ought to be normalising limiting what data we gather in the real world for everything else.

Ultimately, when I ask myself whether performance is more important than user privacy, I end up answering no.

> Ultimately, when I ask myself whether performance is more important than user privacy, I end up answering no.

Thank you for this! We need more people like you building websites. Or less people unlike you, not sure which :)

Like others pointed out, it's fairly easy to test in a "lab" without affecting actual users. I wish more developers did just that to see how their software/website fares on low-resource hardware or lossy connection.

You don't really need to track users. Send back a few timings: LCP, FID, whatever, and nothing else. That's all you need to look at the distribution of real world performance data and compute useful statistics.

You don't need to associate data points with any kind of user profile. Sure, you _could_ maybe mine some noisy timings for user fingerprint entropy but... if you're building something more complicated than a blog, real performance data is very useful and the privacy implication is relatively minimal.

What are LCP and FID?

LCP: Largest Contentful Paint: "render time of the largest image or text block visible within the viewport, relative to when the page first started loading"

https://web.dev/lcp/

FID: First Input Delay: "the time from when a user first interacts with a page to the time when the browser is actually able to begin processing event handlers in response to that interaction"

https://web.dev/fid/

https://web.dev/vitals/

I believe that given that one knows the user agents of users, it is possible to create a test suite with browser automation tools to measure performance in some interactive scenarios in different browsers. It's still 'lab data', but not as crude as just time-to-first-render or whatever you can measure without considering user actions. I doubt the difference with real data will matter at this point. Am I wrong?

For starters, you lab data doesn't account for latency from different locations or client caches.

Good point! But there's an abundance of network tools to simulate latency and packet loss. Last i checked, even my Tor Browser has that in the debugging tools (F12) and can also disable caching.

You don't need to track users to track performance. You only need to know that a page/feature took X ms to load, there is no user-related data there. If you want to add other dimensions like user agent or country, then you may step a bit into user data, but it's as easy as avoid those metrics.

What's funny about this is that client-side "performance metrics" scripts is really close to the definition of malware (only the data points gathered differ with an actual tracker), doesn't work in all cases (older browsers, NoScript..).

But what's really hilarious about it all is that in my experience this kind of sites doing metrics gathering are the worst offenders in terms of performance. Please, just do what you need to do and don't add more bloat in the name of performance.

If you need to do tests, setup a test VM with 512MB RAM, 1s artificial latency and 20% packet loss. By testing for the "worst" case (which is close to the reality of many people) you ensure the average case is great.

> doesn't work in all cases (older browsers, NoScript..)

And that is fine. Even if old browsers and noscript were supported, it's usually a small percentage.

> kind of sites doing metrics gathering are the worst offenders in terms of performance.

You have a point. I have a feeling it's an org problem, bloated org builds bloated products, but struggle to articulate it further.

> setup a test VM with 512MB RAM, 1s artificial latency and 20% packet loss.

A synthetic approach will get you a long way already. Though I would like to see it as part of the CI/CD pipeline. Testing perf has to be passive and continuous or it's useless.

Once you have tracking code in the client the temptation to use it to answer problems will be very tempting, and the pressure from other stakeholders to add 'useful' data will pile on.

If your product manager wants to know user screen resolutions it's much harder for them to argue "We track nothing, we need to implement tracking to get screen resolutions." than it is for them to say "We already track loading times; it's a few additional lines of code to add screen resolutions".

Taking what I think is a pretty white hat use case here: suppose the site is slow for 5% of users. Knowing that a disproportionate number of those getting a slow experience came from country X would be really helpful to diagnosing and fixing. (The point of tracking performance from real users is to improve performance for real users, not to draw pretty graphs.)

>Ultimately, when I ask myself whether performance is more important than user privacy, I end up answering no.

FWIW: In the past when I sold and supported these kinds of tools, users were completely anonymized - you saw them as "requests" rather than users. In fact, the best tools I saw didn't even capture "all" users, but rather did sampling based on various attributes, IE: request was outside the standard deviation.

A beta/early-access program?

Tracking people almost always a bad idea in my opinion.

Even if they get something in return like early access to something.

Even if they're happy to accept being tracked.

There just isn't a good justification for it in most cases. For a start, most sites that track users don't even use the data they gather. They're just gathering it because they're think they're expected to. So many sites implement Google Analytics, Hotjar, etc and then use the data as nothing more than a web counter. That's beyond infuriating to anyone who thinks privacy is worth fighting for. If you're going to violate my privacy, at least do it for a good reason.

In the case of the article there could be solid reasons for hoovering up lots of information about users. But really, unless you're in a massive company with significant resources, all of that perf data is just going to sit in a database and a senior might occasionally look at the dashboard overview and think "We should make our images smaller." or "Huh, users have a sucky time when they're on Android. Oh well.". Those are not good reasons to gather real-world user data instead of relying on Lighthouse scores. Lighthouse is "good enough" for most people.

I imagine that a frustrating part of working on web perf is that small companies and startups don't have the resources to do it well, so they end up cargo-culting what Google does but without spending time or effort to make good on what they might learn.

To be fair to the author, I'm much more at fault here than they are. I have no doubt that the article is aimed it at a much smaller audience (the web perf community), and now I'm here saying it's not applicable in a much broader context because it was posted to HN. I apologize to them for that. This is something I care about and it was an opportunity to rant. :)

I’m not sure we are in agreement on what tracking means.

This article advocates logging performance timings. You can’t track someone with that.

This article advocates logging performance timings. You can’t track someone with that.

Firstly, yes you can - https://www.researchgate.net/publication/2441386_Timing_Atta... - but that's not the point here.

Secondly, web perf testing benefits from user tracking. It is very common for perf and tracking to be used together. In fact, I don't actually know of a time where I've seen perf metrics being used without tracking. I'd agree that it's possible to track perf metrics without tying them to a specific user, but even then you still need to be sending user data back to your server. I'm not in favour of that for something as trivial (in most case) as just making a website load slightly faster.

Site owners should start with the assumption that they don't actually need data and then only track things that are critical to the functioning of their website. There will be very few cases where knowing how fast the page loaded actually meets that criteria.

> Firstly, yes you can

20 year old paper. "Domain tagging," Felten and Schneider's proposed countermeasure, was implemented as "cache partitioning" in Firefox, Safari, and Chrome. If you're aware of extant issues please let me know.

> In fact, I don't actually know of a time where I've seen perf metrics being used without tracking

Google's own library: https://github.com/GoogleChrome/web-vitals#overview, which is recommended in OP's article.

> I'm not in favour of that for something as trivial (in most case) as just making a website load slightly faster

No performance RUM means no knowledge of performance regressions. I have deployed changes that looked great on our CI and then gravely impacted performance for a segment of customers.

Blinding a business to user experience because you've read papers on historic exploits seems like odd logic to me. You can't do anything malicious with a DB of LCP timings.

HN's anti-telemetry mindset is baffling to me.

The trick here is to ask for explicit consent, and use "first party" tracking solutions that run on your own servers instead of phone home. You can probably get perf only tooling as well, for which I don't believe you even need permission (GDPR wise).

It's true GDPR is so broad you don't need to gather consent for such metrics, as it should be covered by contractual obligations (providing a best-effort service) and the legitimate interest of the operator (improving their service), both of which are exemptions to explicit consent.

However, if you want good performance, do actual UX/performance studies without treating your users like lab rats (hell, even poor rats don't deserve that). Just nitpicking, but "first-party tracking" really falls into "phoning home" (just a different "home").

FYI, if you disable timing API used for this in the browser, you lose access to all cloudflare protected websites, because CF will endlessly loop on its "verifying your browser" page. :)

FYI, CloudFlare like most (all?) big corps, hates privacy and will block Tor/VPN users as well as NoScript users. Clear win-win solution: don't use CloudFlare unless you absolutely have to.

I didn't know there is a web performance community. But I deal every day with web sluggishness community.

It's a problem we all encounter. Personally, Tor Browser's Safest mode (no javascript) is the only way i find the web usable these past few years. I can have hundreds of tabs open and still browse in a breeze, but if i dare enable JS in a browser, somewhere somehow one tab will manage to tank my whole machine in under a few hours.

I don't encounter this problem - and have rarely disabled JS. Perhaps having hundreds of tabs open is the cause of your issues? Isn't this what bookmarks are for?

Or maybe you have a higher-end machine than i do? I can tell you it's a common occurrence in my circles where we mostly use second-hand hardware. I remember when JS was not that common it took much less resources to browse the web: now people have gotten used to being unable to have different pages open at the same time. "I should close youtube so discord isn't terribly slow".

Also, i don't always have hundreds of tabs open. But even just following links published on HN regularly pops up an article with simple text and images that for some reason will try to run many scripts eating away all my resources. Tor Browser's Safest mode is perfectly able to run dozens/hundreds of tabs.

If you want to be more technical about it, there's obvious reasons for this. Of course, the client-side scripts are vastly inefficient because they're interpreted or JITed. And the scripts often have more than suboptimal ways to perform basic tasks which arguably could (should?) be declarative HTML/CSS elements which the browser could optimize for. Moreover, a single DOM change triggers redrawing, and some scripts are very keen on editing the DOM.

Then of course we could talk about cryptominers and trackers and whatnot. Some people on this very forum have advertised that it would be acceptable to embed miners in web pages so that clients "pay for" the content. This outlines the high-level problem with client-side scripting on the web: who decides what gets computed on our machines? The declarative model is arguably empowering users whereas client-side scripting (in a client-to-server model like HTTP) is certainly empowering website operators to abuse their users.

My challenge to the web developer community:

Can you write something which works in any browser,

regardless of device age, browser version, connection speed,

with progressive enhancement for more modern features?

In 2014 or so a client with whom I hadn't spoken to a few years contacted me. He told me that he originally wanted to contact me about updating the web app he uses to run his business to support his new iPad. But upon trying it, he was surprised to find that everything just worked! That's because I wrote his control panel in simple HTML with almost no CSS, just clean markup and barely a whiff of Javascript. It even works well enough on his Android phone to not require an update, provided he rotates the screen to landscape.

That, on a page that was designed before the terms "mobile first" or "responsive" ever existed. Because good clean markup was, and remains, good clean markup.

Great work!

No. There has to be a cutoff. "any device age" includes my old HTC Desire HD. And it's downright stupid to optimise for it unless you have a very good reason.

Some features are too practical not to use. And some features are next to impossible to enhance progressively without introducing more bloat. There's not really a gradient between "no wheel" and "wheel".

If that Desire HD is not part of your target audience then there's no reason to pay the price to optimise for it.

You call it stupid...

I call it lazy, insensitive, careless, and perhaps a smidgen of incompetent. :)

I don't know my target audience ahead of time, and I want to be prepared for them, known AND unknown.

Your HTC Desire HD is welcome and supported on all my websites, and the fact that you have one is reason enough for me to support it.

I think it is a really shitty and rude thing to tell the user that their browser, device, configuration, etc. is "not good enough", because most of the time they have no control over it, and you're denying access to the people who need it the most.

When you've got a fast desktop to develop on and a decent salary to switch out your phone every year or two, it may be difficult to relate, but there are millions of people out there using five-year-old devices and struggling extra unnecessarily, and I refuse to add to the problem.

It has the nice benefit most untested devices and platforms also working, as well as nice accommodations for retro-computing enthusiasts.

Works for me. By going one way or the other, we select our own customers.

To me, they're not customers, but readers and humans.

To a certain extent, yes. But there’s a hard stop, if you want to provide a secure experience, when it comes to broken security protocols. And remember that security isn’t just for ecommerce sites, but for privacy at a network level.

Sure, but security isn't everything, there is also accessibility, and I balance them by allowing even plaintext HTTP traffic on non-sensitive websites, because to me it is more important to allow access than to provide so-called "privacy", which IMO means very little with HTTPS anyway.

It depends what you’re hosting. As a basic example, Wikipedia might be a non-sensitive website to most people, but if you’re a teen with questions about your sexual identity, living with homophobic parents, then HTTPS will ensure that they just see wikipedia.org at the router level. The same when I worked at GOV.UK: we were just as likely to offer information on paying your council tax as services for spouses suffering domestic abuse.

If you're an abuse victim desperate for information, and all you have is a device which you found at the bottom of the closet which is five years old and has the clock set wrong, what do you think is better?

Option 1: Being able to connect to a public WiFi network and browse Wikipedia over HTTP.

Option 2: Being "secure" in not being able to access the information at all due to HTTPS limitations?

Why not a challenge to the sysadmins of this world: Can you make sure all machines under your care have a browser not older than (say) 5 years?

3 billion people on this planet cannot afford a new phone or a new desktop every 5 years (to put it mildly).

None of those 3 billion people were ever the target of the products I happened to work on. But 10% of our user bases were limited by their IT departments.

It depends so much on what you're building.

I don't agree. I think while I have a device that's working, it has the right to exist and I have the right to keep using it for 20 years if I want to.

Not everyone agrees, certainly not the majors, but I'm OK with just my little comfortable corner of the Web, which also happens to have no ads, trackers, bloat.js, and so on.

This is generally as simple as writing clean HTML and then using `type="module"` and `nomodule` scripts to differentiate between legacy and modern browsers.

Sure, if you don't want what the ux guys have designed or if you are prepared to give me more people and time

The fact that you are being downvoted to hell for asking this says a lot about the sad state of the HN community.

While google use CWV as a part of pagerank (higher scores get higher ranking) most site owners will demand higher scores. They don't care why their site scores bad, or necessarily want any of those useless javascript trackers and animated widgets removed. They want a higher score and you had better well use every trick you can think of to get them there or they'll pay someone else to do it instead.

This is not a developer issue, this is a capitalism issues. Websites are used to make money, ergo they must rank high in search engines. A google engineer framing this as anything else is simply naive. SEO is a high stakes game, and they are the chief architects that made it so.

Don't get me wrong, personally I am happy that Google have forced sites to follow standards, move to more modern solutions and stop using so much javascript and other cpu and bandwidth hogging resources, but while they also continue to demand tracking of the world they are also contributing to the bloat, and being just a little bit hypocritical: "no bragging about scoring well in lighthouse, but it would be a pity if your scores went down and nobody could find you in a search ... ".

> higher scores get higher ranking

Not exactly the way it works. It's a pass/fail.

> For example, a page with an LCP of 1750 ms (better than the “good” LCP guidance) and another one with 2500 ms (at the “good” guidance) would not be distinguished on the basis of the LCP signal [0]

[0] https://support.google.com/webmasters/thread/104436075/core-...

Lighthouse kind of drives you to add (possibly) useless PWA stuff: offline mode with its manifest.json and a webworker, IIRC.

I think the issue is exactly that: 'When a measure becomes a target, it ceases to be a good measure' (https://en.wikipedia.org/wiki/Goodhart%27s_law).

If you don't specifically optimise for Lighthouse, and then score well in Lighthouse, it's likely a good measure of your site's performance. But if you specifically optimise for Lighthouse and then score well in Lighthouse, it's like practising for a psychiatric examination - it's now measuring your ability to game a test, and not much else.

And Google Lighthouse doesn't like progressive images. You can have a well-optimized progressive JPEG that renders with just 15% of the data, but you get dinged for not using Google WebP, and your LCP is counted only when it fully loads (this is a known bug, and it died a painful death in a bikeshed of what a "contentful" image looks like.)

> your LCP is counted only when it fully loads

Great point. Wonder how it handles things like LQIP

How come it's possible to see visual bands in the main graph [0]? It's clearly using a perceptually uniform modern colormap (magma I think), where might be the artifacts coming from?

[0] https://philipwalton.com/static/lh-cwv-correlation-1400w-654...

Methodology is at: https://discuss.httparchive.org/t/lighthouse-scores-as-predi...

All pages with two 'good' core-web-vital grade, and one 'poor' get exactly 200.

All pages with one 'good' and two 'poors' get exactly 100.

Getting 99/101/201/199 requires a much more specific set of performance characteristics.

Reply on news.ycombinator.com