41 karma · joined April 8, 2020
The intent of my entry was to turn the contest from a “model versus model” into a “market versus model” battle. Or if you like, a battle between the contest community and the community of a different kind that makes options markets efficient. You can call this contest highjacking if you want, or just “exogenous data”.
I just hope those who push back on your article read all the way to the last paragraph. I'll repeat it below. Very well put.
I'm happy to answer technical questions about Prophet from anyone here but again, this is somewhat beside the point, which is...
The requirement that people come to your company knowing how to use piss easy baby tools is an extremely dumb and lazy hiring practice. It is also, unfortunately, a common practice in data science job postings. The aggregate effect of this practice being widespread is that talented people with unusual backgrounds get gatekept out of good paying jobs that they’d be exceptional at. Making fun of the job posting and using Prophet has been compared to gatekeeping. To be clear, the Prophet prerequisite is an actual form of gatekeeping being undertaken by a major company that has actual material impacts on people’s careers. The job post excludes people not based on aptitude, but based on whether they have previous experience and familiarity with a tool they could be introduced to and then master in under 15 minutes. A tweet making fun of the job posting is not gatekeeping. Get over it, LinkedIn clout chasers
I would add that since posting my own less-well worded version of this astonishment I have received numerous DM's from people at large companies who are aghast at the way Prophet is a favorite of management. So whether or not this was a problem at Zillow beyond, say, 2015, it might well be the case elsewhere.
If you think you have a good approach, there's probably a paper in this.
Results will be part of a submitted academic paper. Thanks to all who participate.
The recommendations are based on the Elo ratings for derivative-free optimization packages (see https://lnkd.in/ghgmKfN) which are now quite mature.
The notebook will also compare directly the performance of many different optimization strategies, drawn from disparate libraries, on your objective function(s).
This is a lot faster than trying out nlopt, bobyqa, dlib, nevergrad, pysot, hebo, bayesopt, skopt, ax-platform, shgo, pymoo, hyperopt, optuna, platypus, ultraopt and other Python packages, not to mention variations within, yourself.
A lot faster.
I began writing this post because I was working on integrating Prophet into a Python package I call time machines, which is my attempt to remove some ceremony from the use of forecasting packages and compare them. These power some bots that the prediction network (explained at www.microprediction.com if you are interested). How could I not include the most popular time series package?
I hope you interpret this post as nothing more than an attempt to understand the quizzical performance results, without denying the possible utility of Prophet or its strengths (if nothing else it might be classified as a change-point detection package). I mean seriously, can Prophet really be all that bad? At minimum, all those who downloaded Prophet are casting a vote for interpretability, scalability and good documentation - but perhaps accuracy as well in a manner that is hard to grasp quantitatively.
(from the article)
(from the article)
- The author