I mean this is true for GPT4 too once your code starts getting large enough.

Using APIs can get you a larger external memory, but we still have relatively small context windows these days.