I think Linux dynamic user probes would lend themselves quite well for something like this. Fooling around with bpftrace for a few minutes yielded this one-liner, which catches all the conscious X11 selection accesses (and a few unconscious ones, too :)) I provoked on my desktop during a few minutes of testing:

    sudo bpftrace -e 'uprobe:/usr/lib/libX11.so:XGetSelectionOwner{time("%H:%M:%S ");printf("%-6d\n", pid);}'

I am not familiar with Xlib/X11 at all, so I am not sure which library functions one would be most interested in to explore this much further.

Edit: Replaced `XSetSelectionOwner` with `XGetSelectionOwner` in bpftrace invocation.