If anyone wants to download Books3, it seems to be available here: https://thenose.cc/public/AI/EleutherAI_ThePile_v1/pile_prel...

(We noticed The Pile was recently taken offline, so we hosted it: https://thenose.cc. Apparently Books3 was also a part of The Pile, so feel free to download.)