You don't really need to track users. Send back a few timings: LCP, FID, whatever, and nothing else. That's all you need to look at the distribution of real world performance data and compute useful statistics.
You don't need to associate data points with any kind of user profile. Sure, you _could_ maybe mine some noisy timings for user fingerprint entropy but... if you're building something more complicated than a blog, real performance data is very useful and the privacy implication is relatively minimal.