Lets say you, with your human labor, can use 10 bricks to produce 20 bricks. If you do this every year, your wealth grows. first 10, then 20, then 40, then 80, ect.
In this senario, You can trade with your neighbor and run a 10 brick deficit every year, but you wont exponentially grow your production and wealth. You will have 10 the first year, make 20, trade away 10, then end up where you started. You are sustainable forever, but not growing.
Your house will remain small, and the house of your trading partner will grow ever larger.