Yeah, this is the way to go. For when I have to use the AWS console, I use separate Chrome profiles for each of my environments, so I don't even have cookies, saved passwords, etc. that might relate to other accounts. (I also use separate Chrome themes in each environment and use a CSS manipulator such that my prod console is in a bright-freaking-red Chrome window with a red background. Blue for test, green for dev.)
For my own AWS accounts, I use a read-only account in the profile and, on the very rare occasion I want to get clicky rather than use awscli to frob something, I'll log out and log back in as my writable-to-IAM user. I have a third user that only has IAM read/write privileges if I have to deal with that service. (I haven't sold my clients on this approach, though.)