Machine learning saves us $1.7M a year on document previews
dropbox.tech
dropbox.tech
I know "unlimited" options are abused, but Dropbox isn't unlimited?
I would agree if we talked about Backblaze with "unlimited" backup for next-to-nothing, but this is Dropbox which is rather pricey and also limited to a very specific number of (Tera)Bytes.
If they cannot deliver on their promises, shame on them.
And backend is always S3+Glacier which costs about $25/TB/month for storage and $100/TB for download, so not just that filling up your quota to the brim costs them more than twice as much as the what Dropbox charge for a 2TB quota, but overwriting and retrieving the files is going to seriously hurt their budget. Not your budget, none of your business, but that's how it works, that's why anyone other than them went bankrupt, and that's why they don't try to be nice to advanced users.
Edit: But most importantly: if you sell me "x units of anything" it is not abuse if anyone use whatever they thought they bought.
I'll discuss it if a highly technical user keeps "trashing" it by uploading frequently changed and unique data for no good reason (like in trashing a cache) etc, but ordinary users, filling their quota to max? No way.
I'm tired of constantly having to re-learn where the upsell buttons will appear, so that I can avoid them when I actually use the product that I paid for.
Gloriously boring, no flashy corporate identity. I give them money and the give me file storage/sync. One or two bugs and questionable UI choices, but it just works and I don't see them doing a pointless chain of redesigns or ads.
I'll be online for the next hour or so; happy to answer any questions about ML at Dropbox.
So I wondered: who had the initiative to use ML in riviera, the riviera team or the ML team? How do you collabore between the two teams/worlds (production team and data science team)?
From the beginning the ML team's focus was on providing measurable impact to our Previews stakeholders. One thing that helped us collaborate effectively was being transparent about the process and unknowns of ML (which are different from the constraints of non-ML software engineering). We openly shared our process and results, including experimental outcomes that did not work as well as planned and that we did not roll into production. We also worked closely with Previews to define rollout, monitoring, and maintenance processes that would reduce ops load on their side and provide clear escalation paths should something unexpected happen. Consistent and clear communication helps build trust.
On their side, the Previews team has been an amazing ML partner, and it was a joy to work with them.
What I've seen work better is a product approach, where ML is incorporated as a feature (rarely but possible the centerpiece) of a full solution for an industry that provides a new way of doing something and the value that comes with it. The caveat is that this is hard and takes up front R&D and product market fit research that any product would. It doesn't happen in a series of workshops with representatives from the business.
This Dropbox story is an obvious counterexample, and really looks like the mythical "low hanging fruit" that we always want to identify in ideation workshops. But I'd be careful trying to generalize a process for identifying ML use cases from it.
The organizations range on the maturity level from "We want to use AI, can you help us?" to "We have an internal machine learning and data science team that's overbooked, can you help?" to "We have an internal team, but you worked on [domain] for one of your project and we'd like your expertise".
For the expectations, you can deal with an extremely technical team that tells you: I want something that spits JSON. I'll send your service this payload and I expect this payload. So that's a tiny part.
Sometimes, you have to build everything: data acquisition, develop and train models, make a web application for their domain experts with all the bells and whistles, admin, roles, data management, etc. I wrote about some of the problems we hit here[0].
The point is that, finding these problems is an effort that requires a certain skill/process and goodwill from the clients. We worked on a variety of problems.
1. many organizations dismiss ML solutions without actually trying them. Rather, if one hack week style prototype doesn’t work on the first try, it’s chalked up to “over hyped AI” and never sees the light of day. Organizations that succeed with ML don’t do it that way. Instead they ensure the success criteria are stated up front and measured throughout, so you can see why it didn’t work and iterate for v2 and v3. “We spent a bunch of R&D expense on magic bullet v0 and it didn’t succeed immediately” is a leadership and culture problem - you probably can’t succeed with ML until you fix that.
2. Many companies have no idea how to staff and support ML teams, and go through various cycles of either taking statistical researchers and bogging them down with devops or taking pure backend engineers and letting them do unprofessional hackery with no clarifying product quality ML expert in the loop.
You need a foundation of ML operations / infra support that cleanly separates the responsibilities of devops away from the responsibilities of model research, and you must invest in clear data platform tools that facilitate getting data to these teams.
If an org just figures they can throw an ML team sink or swim into an existing devops environment or they can require an ML team to sort out their own data access, it’s setting ML up for disaster - and again you’ll get a lot of cynics rushing to say it’s failing because ML is just hype, when actually it’s failing due to poor leadership, poor team structure and poor resourcing.
The decision to prewarm is ultimately a product decision to give users a better experience across the many surfaces where they encounter file previews. There are limitations to how fast previews can be generated on the fly, even under optimal conditions (unlimited top-of-the-line hardware, max of 1 request at a time). For instance, optimal on-the-fly preview generation for more expensive files (say, a video that is an hour long or a gigapixel image) can add 10s of seconds to tti -- not good from a user perspective!
Given this constraint, we wanted to optimize where we spend on the preview generation infrastructure without negatively impacting the user experience. We chose to do this with a combination of heuristics and the ML solution described in the article.
No, often it's black. You have to do scene-detection if you want meaningful previews/thumbnails, can't always go random about it.
And the original files are stored in HDD and not SSD/RAM.
Source: I don't work there.
Quick Access: Building a Smart Experience for Google Drive (2017) https://research.google/pubs/pub46184/
and a recent follow up
Improving Recommendation Quality in Google Drive: https://storage.googleapis.com/pub-tools-public-publication-...
I think this may too sanguine for the false negatives in that it ignore latency sensitivity. Generally, batch processing (like preview generation during pre-warming) is cheaper than latency sensitive processing (like preview generation when the user is waiting for it). If you don’t take that into account, you can be misled by your cost metrics.
the service is active but he doesn't do marketing for it. he did talk about it on a podcast a couple months ago https://jpadilla.com/2020/01/10/podcast-djangochat-ep-45/
How much will the false negatives cost in customer satisfaction?
How many FTEs will you need to maintain and improve this?
How many more customers did you get from the PR?