I've been using abook [1], which is very old software but is fast, simple, stores data in plain text, and can export to vcard when needed.

[1]: https://abook.sourceforge.io/