How many GB of data are 320 million (HTML) pages? How long does it take to refresh the index (with a single crawler on a 1GB/s connection)?
The incoming data to the crawler... not so sure... something on the order of 10-20tb. I didn't really measure this. And I don't keep all the data. There is no "cache" function.
On a 1gbit/s connection it takes about a week to crawl and generate the index.