Not exactly. In math, epsilon is basically "it's not zero, except when it is."

When you're dividing, then it's close to zero, but still qualitatively not zero. But when you're multiplying or adding, then we say it's close enough that we'll treat it as exactly zero. So it's not a specific number, it's something where we can handwave its qualitative size depending on what's convenient.

Limits are a formalization of this concept. It's not a specific value, it's very much a variable that "approaches" zero. This establishes rules about how much bullshit you can actually do it. It's not zero, you do some arithmetic, then it becomes zero and you do the rest of the arithmetic.

But when you're writing a program, epsilon is indeed often just a small number. The trick is choosing the right "small."