A buddy just PoC'd a key binding system for OS X to do arbitrary complex remappings like this (exactly to facilitate nonstandard esc, ctrl, hyper usage), represented as one or more state machines similar to this syntax:

  STATE1:
    caps down:
      ctrl+alt+cmd down
      STATE2
  STATE2:
    caps up:
      ctrl+alt+cmd up
      press esc
      STATE1
    after 1000ms, default:
      ctrl+alt+cmd up
      STATE1

This would allow us to layer, say, half qwerty on top of hyper with the same system. Or map caps to esc and ctrl, and map caps+shift to hyper, or a non-meta like tab to hyper.

Also could do state transitions on external events (like new app focused), and execute templated applescript.

An easy state machine to build would be this, which would prevent keypresses after focus stealing:

  STATE1:
    new window focused: STATE2
  STATE2:
    after 250ms: STATE1
    default:
      drop keypress
      STATE2