Ah, it looks like their FAQ also says this is supported:
Can I use the same Security Key with multiple Google Accounts?
Yes. You can register the same Security Key with multiple Google Accounts.