I'm not a Pandas person - is it possible to install the Pandas timestamp functionality as a smaller library, or do you need to install the whole shebang?
Generally every other library works in terms of standard python types where possible. Very few other places will you see things like a re-implementation of a time library.
...and scipy and all the scikit libraries and whole bunch of others. 'Accidentally' pulling in hundreds of MBs of libraries you never use for a 100 line data processing script is par for the course in python land.