Doesn't this open you up to replay attacks though? Since you can't store that a token was already used..
AFAIK, neither signatures or "something you have, know" alone fixes replay attacks. Since this is a well known problem in cryptography, many solutions exists. All of which are probably overkill for this use.