This. Why you would leverage system python is beyond me.
Anaconda distributions aren't exactly small. I'd also assume there's some sort of environment isolation, so you're getting multiple copies. Maybe one per workbook?
The embeddable package from python.org [1] is 20 MB unpacked. To install different sets of packages per workbook, just have Excel automatically set the module path as appropriate.