Dear AI startups: Your ML models are dying quietly

Good point. Or perhaps it was that nobody had ownership over the recommendation system. There should've been a performance indicator on a dashboard somewhere.

The problem with numerical programming is that all data is the same type. If you access wrong row, multiply across wrong axis, function uses wrong parameters, it's semantically wrong, but it can still work and system can learn something.

Thanks for your feedback ohazi! You are right that “No system should ever fail silently” and there are many ways to address this. However, we’ve seen cases where not enough attention is put into input data pipelines and similar mistakes are being made by new and even well-funded AI startups. One of the reasons this is happening is because some of these startups are launched by people with little experience in machine learning / data science. For example, we’ve seen doctors, web developers, lawyers and accountants moving into AI space. They can learn the basics of machine learning and maybe hire someone to help them, but often don’t know about the best practices or things to watch out for when building a data driven company. Unfortunately, there are few guidelines or best practices available online. With this article, we were hoping to help these new AI startups by bringing awareness to the importance of data pipelines.

That's true! Thanks for sharing the paper. We'll take a look

In this article, we showed a scenario where a small change to the front-facing interface could dramatically reduce the accuracy/performance of a machine learning model powering the application

I figured that it was just for illustration, because the author couldn't think of a better example. Some real-life examples that turn up stupidly often:

1. The model uses click-through data as an input. Your frontend engineer moves the UI element being clicked upon to a different portion of the page for a certain category of results. This changes the baseline click-through rate. The model assumed this feature had a constant baseline across all results, so the new feature value now needs to be scaled to account for the different user behavior. Nobody thinks to do this.

2. The frontend engineer removes a seemingly-wasted HTTP fetch to reduce latency. This fetch was actually being used to calibrate latency across different datacenters, and was a crucial input to a data pipeline to a system of servers (feeding the ML model) that the frontend team didn't control and wasn't aware of.

3. The frontend engineer accidentally triggers a browser bug in IE7 (gimme a break, it was 9 years ago) that prevents clicks from registering when RTL text is mixed with LTR. Click-through rates decline precipitously in Arabic-speaking nations. This is interpreted by an ML model as all results being poorly performing in Arabic countries, so it promptly starts cycling through results, killing ones that had shown up before with no clicks.

4. A fiber cable is cut across the Pacific. This results in high latency for all Chinese users, which makes them abandon their sessions. A ML model interprets this as Chinese people being less interested in the news headlines of that day.

5. A ML model for detecting abusive traffic uses spikes in the volume of searches for any one single query over short periods of time as a signal. Michael Jackson dies. The model flags everyone searching for him as a bot.

6. A ML model for search suggestions uses follow-up queries as a signal. The NYTimes crossword puzzle comes out. Everybody goes down the list of clues and Googles them. Suddenly, [houston baseball player] suggests [bird sound] as related.

Your startup looks quite interesting and I can actually see potential for commercial usage. How's the growth rate been?