When not using tmux/screen, the complete invocation is C-a a M-x roll-eyes RET Backspace.
Moreover, I forgot if it's the default or if I set it somehow, but in Emacs a single C-a goes back to the beginning of a line, but C-a C-a goes back to the beginning of text on the line, which is almost always what I want anyway. So I press C-a twice anyway, everywhere, and I don't have the problem you described at all :)