I've fixed that with secure boot (my own keys, not Microsoft's) and TPM2. From that host I can run a program named Tang, which operates in conjunction with another program named Clevis.
On hosts without secure boot such as RPis and other ARM SBCs, Clevis will contact Tang at boot time and decrypt the disk.
Incidentally my x86 with secure boot has intel vPro so the KVM is not needed. The remaining sisters have a piece of lost technology used for decades to solve the KVM problem: serial (UART specifically) connections.
Some seasoned sysadmins might have memories of dialing their Sun servers serial ports for remote administration.