For the cryptography & theory? https://toc.cryptobook.us/

For the design and internals of hash functions? The finalists for the SHA3 competition have extensive design documentation. There's an archive at https://web.archive.org/web/20170829225940/http://csrc.nist....

Cryptographic hash functions are designed to resist existing attacks, so you'll want an understanding of differential & linear cryptanalysis, as well as a variety of algebraic attacks. I don't know of a good textbook on the subject, so you might find yourself searching keywords on https://eprint.iacr.org/