Current Time Series Anomaly Detection Benchmarks Are Flawed
arxiv.org
arxiv.org
The following quote from the article is striking: "Qiu et al. introduce a “novel anomaly detector for time-series KPIs based on supervised deep-learning models with convolution and long short-term memory (LSTM) neural networks, and a variational auto-encoder (VAE) oversampling model.” This description sounds like it has many “moving parts”, and indeed, the dozen or so explicitly listed parameters include: convolution filter, activation, kernel size, strides, padding, LSTM input size, dense input size, softmax loss function, window size, learning rate and batch size. All of this is to demonstrate “accuracy exceeding 0.90.” However, as we will show, much of the results of this complex approach can be duplicated with a single line of code and a few minutes of effort."
Much of CS is like this; useless papers with incremental improvements at best. Without having read the paper, I'd guess plain ARIMA etc. do well enough in univariate forecasts.
There is also a video here [b]
[a] https://www.dropbox.com/scl/fi/cwduv5idkwx9ci328nfpy/Problem...
A paper published today finds an anomaly in an "InternalBleeding" dataset, after setting eighteen parameters [a].
Could we find the anomaly with a completely parameter-free algorithm? As the figure below shows, the answer is YES, if you use MADRID [b].
One line of code >>MADRID(UCRAnomalyInternalBleeding)
So Yes, simple is better.
[a] Learning Rate, Dropout Rate, Dim Feedforward, Batch Size, Encoder Layers, Decoder Layers, Activation Func, Time Warping, Time Masking, Gaussian Noise, Linear Embedding, Phase Type, Self Conditioning, Layer Norm, Pos. Enc. Type, FFN Layers, Window Size.
[b] https://www.dropbox.com/scl/fi/hd9gt0xs8v8mrsx3upwd3/ICDM23_...