Infinity is weird - what's bigger than big?

Regrettably, that's a reply that's often used to dodge the question.

Infinity is not a number in the sense of existing on the real number line. However, we can define a limited arithmetic using infinities of two different types: ordinal and cardinal. In doing so we have to lose some things (like subtraction, division, and in some cases, commutativity) but in return we do get functioning algebraic systems.

So your teacher was basically avoiding the problems by saying "Thinking of infinity as a number leads to problems, so let's never talk about it." In doing so they shut down the conversation.

In some sense, that's a shame.