Do LLM not also think when they encode the prompt? If Karpathy's explanation is accurate, longer prompts should also help even if they don't contain additional information, just by virtue of giving more time to think.
So no, if you just fill up your context window to garbage, the LLM will not perform better at your task/question.