If you play with a "raw" model such as LLaMA you'll find what you suggest is true. These models do what you'd expect of a model that was trained to predict the next token.

It's quite tricky to convince such a model to do what you want. You have to conceptualize it and then imagine an optimal prefix leading to the sort of output you've conceptualized. That said, people discovered some fairly general-purpose prefixes, e.g.

    Q: What is the 3rd law of Thermodynamics?
    A: 

This inspired the idea of "instruct tuning" of LLMs where fine-tuning techniques are applied to "raw" models to make them more amenable to completion of scripts where instructions are provided in a preamble and then examples of executions of those instructions follow.

This ends up being way more convenient. Now all the prompter has to do is conceptualize what they want and expect that the LLM will receive it as instruction. It simplifies prompting and makes the LLM more steerable, more useful, more helpful.

This is further refined through the use of explicit {:user}, {:assistant}, and {:system} tags which divide LLM contexts into different segments with explicit interpretations of the meaning of each segment. This is where "chat instruction" arises in models such as GPT-3.5.