In the 20th century, trainspotters, radio hams, people who built toy model railways or had toy soldiers, science fiction fans, chess players come to mind. See also https://en.wikipedia.org/wiki/Anorak_(slang)
Steven Levy's book, Hackers: Heroes of the Computer Revolution, tells the story of how in the late 1950s the Tech Model Railroad Club at MIT morphed into a computer club around the TX-0 and produced a number of early influential computer scientists and engineers.