The actually TeX binary is actually quite small - it is little more than a macro processor. The other packages could actually be loaded on demand and kept in a local cache.