someone here is writing such a book in public, here's the thread with the discussion

"Implementing a ChatGPT-like LLM from scratch, step by step"

https://news.ycombinator.com/item?id=39156778

you can view the github for the book as they work on it https://github.com/rasbt/LLMs-from-scratch