It's a dwindling resource in the sense that it's not renewable (on human timescales, anyway) or reusable: once it's in the atmosphere, it's effectively gone.
And unlike fossil fuels that are (likely) only found on Earth, Helium is the second most common element in the universe and can be found literally everywhere once you're off planet. Plus we can technically manufacture it from radioactive decay.
In the vacuum of space, not enough to be harvested. Presumably the same for asteroids, moons and planets.
Stars have lots and lots of helium, but good luck mining it.