Ask HN: Strategy for Dealing with 0.5PB of Data and AWS

2 pointsesalman2 comments

I work at an Neuroimaging lab which is in the middle of a transition. We have about 0.5PB of data which may grow to 1PB in next 1-1.5 years. We have limited funding to utilize AWS services to store the data.

At any time we need to have maybe around 100TB of this data be available locally for computation at a high performance computing cluster. We have ample local storage.

Funding is a big concern, so what kind of solution (based on AWS) will be most cost-effective for this scenario? Do we need to think about offsite back-up when AWS is available? Is there a resource or case study which we can learn from?

View on HN