lol they didn’t open source the model weights
https://storage.googleapis.com/books/ngrams/books/datasetsv3...
(I don't remember exactly how much it is, but I remember that the old version was already in the terabytes.)
I want to see how well weights for these models compress, but it will take me some time to run this code and generate some. I'm guessing they won't compress well, but I can't articulate a reason why.
This model has 3.12TB of floats??? That's insane. How do you load that into memory for inferencing?
Alternatively order something like the HP Z8 with 3TB RAM configured, which is only $75k - https://zworkstations.com/configurations/2040422/
It's interesting. It would take ~six years for the Z8 to break even compared to AWS, but traffic into and out of the machine would be $0, and I don't think you're running directly on the metal with AWS, so performance would probably be a bit higher. And then there's storage - I configured, uhh, 120TB of a mixture of SSDs and HDDs. I'm not even going to try and ask AWS for a comparible quote there.
I may or may not have added dual Xeon Platinum 8280s to the Z8 as well. :P
Do you mean six months?
Yup.
Also - think you meant 6 months, not 6 years anyhow :)
And I did mean 6 months, woops. Didn't even notice...