A hole at any of those points could represent either a one or zero
That's not quite correct. There were twelve rows: 0-9, 11 and 12. A hole in row 0 was a zero, etc. Holes in 12 + 1 was an 'A', 11 + 1 was 'J', and 0 + 2 was 'S'. Other combinations were for punctuation.