Ask HN: Why don't LLMs encode sentences to get longer context?

I am but a simple web code farmer, but playing with LLMs for RAG etc.

It’s not possible to give a whole book to ChatGPT or much close to it.

But what if the tokenizer just tokenized whole sentences into a single embedding and passed those instead?

How bad would the data loss be?