I'd been considering something similar, never really made a PoC though.
How about this: User provides their public key when they signup. To authenticate, server produces a nonce and sends to the client, client signs the nonce with private key and sends the result, server verifies signature with public key.