<COMPOSE> " a

where COMPOSE is typically ALTGR, but you can map it anywhere you want, like CAPSLOCK

I write daily in three european languages with many accented characters, and this is enough for all of them. I have even mapped composing a to α, b to β and so on, which is useful for the (occasional) greek word.

EDIT: (thanks to gpvos comment below), on linux with xorg you can achieve this by the following configuration:

   # setup base keyboard layoud and model
   setxkbmap us
   setxkbmap -model evdev

   # set compose to right alt
   setxkbmap -option compose:ralt

   # disable capslock sticky effect, set it as modifier
   setxkbmap -option caps:ctrl_modifier
   xkbset nullify lock

   # enable dead greek letters
   xmodmap -e "keycode 148 = dead_greek dead_greek dead_greek dead_greek"
   xmodmap -e "keycode 66 = dead_greek dead_greek dead_greek dead_greek"

   # swap brackets with parens (useful for programming)
   xmodmap -e "keycode 18 = 9 bracketleft"
   xmodmap -e "keycode 19 = 0 bracketright" 
   xmodmap -e "keycode 34 = parenleft braceleft"
   xmodmap -e "keycode 35 = parenright braceright"

   # enable ridiculously fast repeat
   xset r rate 250 120