I'm not saying that you need to grant any kind of permission in order to use U2F tokens, but rather that a user thinking "I want to login to Google" and "I need to use that USB key thingy to do that" is quite likely to accept a prompt that requests access to the U2F device.