[0] There is also the growth that comes in spurts from technological innovations. Those spurts should be amortized across the entire ranges of years that follow them when assessing the resultant growth. IMO, such technological innovations are akin to data compression, increasing entropy. Over the loooong haul, I'd expect such spurts to have diminishing returns (just like compressing a compressed file has diminishing returns.)
[1] I guess expanding our reach beyond Earth would be one of those 'spurts'.
EDIT: fixed a word