The most technically challenging part was probably wiring the storage & sharing side up to not behave like a traditional Dropbox shared folder or shared link (e.g. not take up storage quota or sync down to your machine). There were certainly a long tail of internal reliability work to get that operating smoothly (there are many internal services to work with here).

In terms of tricks at handling failures:

The first (and probably most important) thing is finding the failures. Bug bashing sessions and also just reading failure logs can help catch the trickier transient problems our integration or unit tests do not catch.

In terms of mitigating them, retries and correct error handlers in code are essential (e.g. Promise.catch case should not be ignored!). Retries on the client side are made a bit easier as we chunk uploads into 4mb blocks so any upload errors of actual content can be retried piecemeal.