Live Speech: I actually answer unknown phone numbers (usually) and would like text-to-speech on my calls because I've started to get concerned about what can be done, fraud-wise, with even small samples of my voice. So in this case, using another's voice is fine, even preferred. (Edit: I suppose I'd actually prefer a voice-changer here, which is less related to this accessibility feature. But I think Apple is unlikely to do that.)
Personal Voice: When my girlfriend texts me when I'm out with my AirPods, I think we'd both like me to hear her message in her actual voice rather than Siri's. This feature doesn't allow for that yet, but the pieces are all there.
Finally, Apple needs to detect and tell me when I'm listening to a synthetic voice, including when it's not being generated by Apple. There's fraud potential here. I'm clearly excited about this tech, but I want to know more on this front.