Ask HN: How are LLM's served at scale?

How do LLM servers handle contextual data? Is the context passed on a prefix to a stateless machine? (Would mean a lot of tokens have to reprocessed during a session) Or a separate LLM instance is created and maintained for an active session? (Expensive and inefficient)