Nice! I initially wanted to build a similar device using just an ATtiny85 that speaks USB using USBtiny [1] or V-USB [2]. It would be low-cost but also not secure. Using a crypto processor like the ATECC508A is obviously a saner choice.

[1] http://dicks.home.xs4all.nl/avr/usbtiny/ [2] https://www.obdev.at/products/vusb/index.html