Just tried this, you are correct... http://imgur.com/J9tP7lk (note the incorrect bracket highlighting)
Lexing and producing tokens with a loop is probably enough given that auto-indentation works on a subset of states of the parser, if any. As my comment alluded to, using regex is harder to both write and read than a simple loop; it's also most likely slower.
It terrifies me to think about how these language modes contain regexps for matching regexp syntax. Of course those are wrong.
You need a full parser for that to happen, most mode authors don't bother[0]. JS2-mode (https://github.com/mooz/js2-mode/) does that (or attempts to).
Most syntactic colorisers are souped-up tokenisers, not actual parsers. It suffices in 99% of cases, but breaks badly elsewhere.
[0] it would help if more languages made fast parsing machinery externally available as a convenient API. ALAS, that's rarely the case, even less so for fast parsing machinery which doesn't throw the information you need for syntactic coloration out the window.
https://github.com/mooz/js2-mode/issues
The goal just shifts from having a semi-accurate regexp tokenizer, to having the language mode's complex parser match the complex parser of the language it targets—which will pretty much never happen, unless the language is exquisitely simple, like maybe restricted dialects of Scheme... or Brainfuck.
It would be easier with a more stable language. I.e. not JavaScript, which is still going through growing pains.
Even so, "25 Open, 204 Closed" seems pretty good to me.
Emacs has a generic lightweight parsing facility that can tell you if you're inside a string, or a comment, or how to skip over a string, or a pair of matching parens/braces/brackets, and so on.