If you're keen to go down the wikipedia hole,
https://en.wikipedia.org/wiki/Six-bit_character_code and then
https://en.wikipedia.org/wiki/BCD_(character_encoding) explain that IBM created a 6-bit card punch encoding for alphanumeric data in 1928, that this code was adopted by other manufacturers, and that IBM's early electronic computers' word sizes were based on that code. (Hazarding a guess, but perhaps to take advantage of existing manufacturing processes for card-handling hardware, or for compatibility with customers existing card handling equipment, teletypes, etc.)
So backward compatibility is likely the most historically accurate answer. Fewer bits wouldn't have been compatible, more bits might not have been usable!
(Why 6 bit codes for punch cards in 1928? Dunno. Perhaps merely the physical properties of paper cards and the hardware for reading them. This article talks about that stuff: https://web.archive.org/web/20120511034402/http://www.ieeegh...)