No, you haven't. You've seen some symbols that represent the ludicrous physical notion of a negative amount of an object. Look outside, count negative 3 sheep. You can't. It's a _fiction_ which happens to be useful for accounting for debt.
Infinities are quite different, given that it is by definition impossible to measure whether something in reality is infinite or not unless you have infinite time.
It's impossible to count a negative number of things. It's impossible to count an imaginary number of things, it's impossible to count a matrix of things. It's impossible to count an irrational number of things. These are all fictions that people use because they're useful. Infinity is exactly the same as any of those other concepts.
Also I clearly just demonstrated that you can count negative numbers easily so I don't understand why you find it impossible.