Programming languages are more general than the problems they solve. (= feature, not bug)
Still, yes, you can mess up your "add 1 to the input" program and make it run infinitely.
Still, yes, you can mess up your "add 1 to the input" program and make it run infinitely.
That being said, regexps were not initially meant to be "programming languages", so I'm not sure about the "feature, not bug" part. I'd rather have a notation that would let me solve, for instance, the "HTML tag matching" problem and would be guaranteed to always terminate, than one that also lets me implement Conway's game of life.