Pwn the ESP32 Forever: Flash Encryption and Secure Boot Keys Extraction

I don't quite understand why the author considers the ESP 32 a "broken platform" now that this attack was possible.

Doesn't it definitely require physical access? If so, couldn't you just swap the whole device for a malicious one? Is the issue that you can read out secret keys burned into the firmware?