To counteract brute-forcing, how about adding a rate-limiting mechanism?

I could imagine each attempt to rotate the cylinder could partially compress a spring-loaded lever. There could be some sort of ratcheting return mechanism that allows the spring to decompress at a known rate (think a kitchen timer). Once the spring is compressed beyond a certain point (e.g. after 5 failed attempts), a mechanism locks out the cylinder until the return mechanism allows the spring to decompress back to its starting position.

If the lockout happened after 5 failed attempts, and the lockout duration was 10 minutes, an attacker could test at a maximum rate of 30 combinations per hour. It would take 64 days to check all 46,656 possible combinations, or 32 days on average to find the solution.