I'd say: it really doesn't matter at all, in fact it's even more argument to make the pointer as low latency as possible. The mouse pointer is your control element as the controller of the interface. The UI element (and everything else onscreen) is the computer's element. It is an interaction between two entities and there is no need to introduce an expectation that the computer be "zero frame delay" on UI elements. It certainly sucks if it's more than a few frames for it to react, but so be it. However for the mouse cursor itself, it should absolutely be immediate because it is the user's representation on this virtual surface, and as such should follow inputs without being unnecessarily impeded. (Case in point: some UIs intentionally add delay or effects [like a fade-in] to UI element response. It's a… "choice"… but sometimes it works.)

Ultimately, the mouse pointer is an extension of hand-eye coordination; the only thing worse than making it lag (which you can learn to adapt to, albeit it will be very annoying) is to make it lag inconsistently (which you can't adapt to and will grow frustrated with extremely quickly.)

And, unfortunately, from personal experience, the mouse pointer on Wayland gets choppy if system load is high. X11 was significantly better with this; with Wayland the compositor [kwin_wayland to be specific, might be a KDE issue] doesn't even seem to be "unnice" or have some kind of scheduling priority over other tasks — I'm not sure if I'm missing something there but that seems to be a glaring oversight on its own.