Hacker News Reader

Comment by lgkk | Hacker News Reader

How do you move that much data over to another cloud provider?

Without losing data or disrupting the customer?

Or do the databases just stay in the data center and not migrate.

Roughly, pick a date and start writing all new data to both, while running etls to backfill data from old to new. Once that is done, you use a feature flag to do a small % of reads from the new system and wait and pray. If nothing major pops up, you slowly ramp up the % of reads until you're confident (as much as one can be) that everything is working, then you move 100% of reads to the new system. Finally, you turn off writes to the old system, clean up the feature flag and remove any old unneeded code.

jk of course you never do any of the other stuff after turn off writes to the old system you just leave the FF turned on to 100% and never think about it again :-p

I did a migration like this before intercloud between es clusters. Just wanted to additionally confirm that this is broadly the way that I've seen it done before.

If you can afford downtime, or data loss that does make it easier. But that's not an engineering question, more a product one.

We (when I worked at LinkedIn) did it with ETL clusters, we already had built them out for moving data between datacenters nightly. They would mirror an HDFS cluster, then ran batch jobs to transfer either directly to the outbound cluster or to another ETL cluster in another DC.

We used one of our ETL clusters to ship data to MSFT for various LinkedIn integrations, like seeing LinkedIn profile information in Outlook or Office products.

Which tools were you using for ETL? Or were they completely custom?

Live replicas (perhaps initialized with a cold backup, initially, if the dataset's really huge), carving off parts of it for separate migration if that's at all feasible, and some expensive folks doing a lot of butt-clenching-worthy activity for an hour or two (unless it goes very poorly...) for the final cut-over, some evening.

Reply on news.ycombinator.com