It's possible to write 256 bit of user data to the Pis internal OTP memory. You can store a non-changeable key there. Of course that's then easily readable if you can run execute commands on the Pi.

LittleFS (recently posted here) is indeed interesting, although from the documentation I couldn't figure out

1) if it works for larger devices as it seems it seems mostly intended for sizes of a few MB. I might be totally wrong with that.

2) if/how it handles corruption of complete blocks(?) of SD memory. As far as I understand, an SD card doesn't necessarily use 4K block sizes and corruption might hit multiple blocks at once. If LittleFS stores both previous and next version next to each other, I don't think that'll help in that case.