That's a great point, listening and speaking is much harder than just being able to read and match some words.
We did consider using text-to-speech to say the word out loud once you answer it, but it wouldn't be ideal to have to hear every time you wanted to unlock your phone.