I thought the same thing reading this! I'm currently reading Prisoner's Dilemma by William Poundstone, which details the life of John von Neumann. According to his brother Nicholas, von Neumann likely got the idea to use punch cards for computers from the Jacquard loom. One of his father's (Maxwell von Neumann) clients at the turn of the century was a Jacquard loom manufacturer, and Max would evidently discuss his work with his kids everyday over supper.