You can embed a minimal ssh server (like dropbear or tinyssh) in your initramfs to remotely enter the passphrase. There are some guides for that online