There's no bar of entry whatsoever for this. Chrome will automatically update itself to Chrome 94 and voila - it tracks your idle status. Worse yet is if you try to complain, they can shift the conversation by saying the ability is there for websites behind a permission dialog, and of course they take user privacy seriously. Basically exactly what you've done!
I don't think it's paranoid to assume nefarious intent because web devs weren't asking for this functionality nor was it impossible before. You add a mouse listener to the page, you check document.hidden, etc. With so many techniques already available ask yourself: why did Google spend time building this? And why didn't they consult other web stakeholders?