Edit: Google's Vault [0] is a custom hardware implementation of this concept.
[0] https://techcrunch.com/2015/05/29/googles-project-vault-is-a...
[edit] But then I hadn't thought about legacy devices.
Minor nitpick:
signed and encrypted using that public key
You don't sign things with a public key, it's your private key you sign with.
You can also always plug the camera to a laptop and use it as a capture device basically.
That's if you want "one-way" encryption (i.e. once the photo is taken you can't retrieve it again until the private key is present).
The private key needs to be in memory as long as you are filming/camera is on since otherwise you can't really encrypt the data so you can't film.
If you want to decrypt the data, you need the private key corresponding to the public key that this was encrypted with, but that can be left at home, but you never need it before that, so you can leave it at home if you want and you won't be able to access it until you get home.
A symmetric key is 256 bits, an HD video is on the order of hundreds of millions to billions of bits, so even if your public key crypto system were 100,000x slower than the symmetric crypto system, it's still going to be negligible compared to the cost of decrypting the remainder of the data. See https://en.wikipedia.org/wiki/Hybrid_cryptosystem
For this reason, public/private key systems for data at rest is probably not particularly ineffcient relative to your suggestion of a symmetric FDE, plus it has the additional benefit that if you want to operate without the ability to decrypt data at rest, you have that option (by leaving the private key at home, since you only need the public key to write new data).
It's true that TLS as implemented on the internet encrypts with a private key and decrypts with a public key, but this is not universal. Encrypted PGP communications, for example, work the opposite way.
Encrypting with a public key is a common technique if you want to ensure that only one person or device can decrypt it, which is what is desired here. It doesn't prove authenticity, unless you also sign the message with your own private key, but that's not the goal -- protection of the message from unwanted actors (and yourself even under compulsion) is the goal.