Longer context means more memory, effectively a longer history the LLM remembers. One issue i'm having is say functions works wonderfully, but context window is tight even with 16k tokens, with a bigger context, sky is the limit.
However, a longer context does mean you can give a lot more verbose instruction, maybe feed a whole code base over just a single script to give wider context over the current problem