If anyone's interested, I think this is the github url of this project (had to dig a bit since it's not mentioned in the page): https://github.com/sjvasquez/handwriting-synthesis
alphabet = [ '\x00', ' ', '!', '"', '#', "'", '(', ')', ',', '-', '.', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', ':', ';', '?', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'R', 'S', 'T', 'U', 'V', 'W', 'Y', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z' ]
That seems to explain why it fails on capital X/Q and doesn't fail on lowercase x/q.
Seems like these characters were not present in training data