I would argue that this statement is a mistake that should have been foreseen:
"negative integers isn't literals!"
Specifically, twos complement signed integers will be a bitch of you continue down that path.
On the other hand, having negative integers as literals will cause even harder problems with subtraction (as in "2-2")
In a language like C the parser is going to try to parse Expr - Expr but not Expr Expr since there's no such construct in the language to put two expressions after each other.
Actually, it's an incredibly good idea, cause you can then just implement constant folding on the AST, and that (unary negate)1 turns into an actual -1, as well as a lot of other good optimizations.