I'm not sure why you think the underlying physics is deterministic. That idea was the state of the art understanding in say the 19th century (billiard balls and Newtonian physics), but quantum mechanics, at the beginning of the 20th century, fundamentally changed the understanding of reality at the lowest level.

For a good layman's introduction I would suggest, "QED The Strange Theory of Light and Matter", (1986) by Feynman. Hard ideas but clear prose. I like to read it every few years to remind myself of the fundamentals.