It's worth noting that much of his work at the patent office related to questions about transmission of electric signals and electrical–mechanical synchronization of time, two technical problems that show up conspicuously in the thought experiments that eventually led Einstein to his radical conclusions about the nature of light and the fundamental connection between space and time.

The special theory (among other goodies), but I am willing to bet nobody was investing more quality time at that problem at that year(Theoretical physics was a game played in a few European cities). For the General Theory he spent 10 years (1905-1915), he worked at the patent office until 1909, and even yet, he got almost scooped by Hilbert.