If the aphelion (furthest point of its orbit) is significantly farther out than the Oort Cloud, then it fundamentally can’t be from anywhere inside our solar system per orbital mechanics. And in this case, its “orbit” is hyperbolic. Meaning it’s going faster than the escape velocity, so will never return.

It’s also a dead giveaway if the eccentricity or direction of its orbit are significantly different than those of the rest of the solar system

Edit: For objects as far out as the Oort Cloud, a few relatively small changes in velocity can actually result in hyperbolic orbits. The real distinction is a big difference in velocity.