Mathematics agrees with you, there are in fact infinitely many numbers.

Physics and reality do not. Once the number gets big enough it loses all practical value. Humans are not really built to reason about very large numbers. At some point you reach Zimbabwean inflation and your currency loses all value because everyone is paid in the trillions. Then you reset your currency back to lower numbers because that's what humans and their systems can handle.

So, no, the number cannot always go up. There is a practical ceiling beyond which numbers lose meaning and value.