In a code review a couple of years ago, I had to say "no" to a dev casually including pandas (and in turn numpy) for a one-liner convenience function in a Django web app that has no involvement with any number crunching whatsoever.
Unless is was absolutely critical the server have as small as a footprint as humanly possible and it was absolutely guaranteed there would never need to be included in the future of course. However, that first constraint is the main one.