In this case, the question assumes that an attacker would more or less be creating their own database of hashes and derived keys (to search for and decrypt known photos and associate them with user accounts, or to bruteforce unknown photos), and would therefore have no need to worry about acquiring the key used for blinding the CSAM hash database.
Decrypting vouchers requires the server blinding key and the NeuralHash derived metadata of the input image (technical summary page 10, Bellare Fig. 1 line 18). This attacker only has the latter.
In the text you referenced, it specifically says that the blinding key would be needed to decrypt vouchers which are CSAM matches. This is because Apple set up their CSAM database in a blinded manner. Therefore to access a hash from the database from which to derive a decryption key, Apple would need the blinding key to first decrypt that hash value.
However, and attacker would be generating their own (presumably unblinded) database, and therefore wouldn't need to access Apple's blinding key.
Apple needs the blinding key, because they encrypt their list of NeuralHashes hashes first, so that others cannot see exactly which CSAM hashes they're testing against. Therefore they first need to decrypt their own database in order to get the corresponding hash value from which to derive the decryption key.
> Next, the client creates a cryptographic safety voucher that has the following properties: If the user image hash matches the entry in the known CSAM hash list, then the NeuralHash of the user image exactly transforms to the blinded hash if it went through the series of transformations done at database setup time. Based on this property, the server will be able to use the cryptographic header (derived from the NeuralHash) and using the server-side secret, can compute the derived encryption key and successfully decrypt the associated payload data.
Is that correct?
If so, then I agree that it is true that in the PSI system the server secret is completely necessary as part of the decryption process in order to decrypt the matching hash in the pointed-to location in the table. That being said, looking only at the information encrypted by the client, I don't think the server secret comes into play, right?
If I'm misunderstanding, and you're confident that an attacker would have to have the server secret to decrypt a photo (even if they already knew that photo's NeuralHash and were able to defeat the internal layer of encryption), then I definitely recommend posting a well-outlined answer to the Cryptography Stack Exchange, as that would be super helpful!
It’s not useful to talk about defeating the “internal” layer of encryption separately from the “outer” layer because vouchers should be stored as generated by clients at rest. The database leak should not include any unwrapped vouchers (that would be like using a password hash but storing plaintext passwords anyways).