I was 11 or 12. I learned the really hard way. I was in Norway for a month and I only had an old 486 at my disposal, a hex editor and some Sega Megadrive games and an emulator (the Megadrive uses a Motorola 68000 processor)
I was curious how the games worked so I started hex editing them at random, seeing what would happen if I changed this and that, etc.
Long story short I learned to write programs in 68000 machine code (binary) and recognize patterns in ROMs (music, art, code). I still remember how to code simple things in machine code.
Imagine my joy when I discovered assemblers, and after that, compilers a while later :)