Hacker News Reader

Comment by cqqxo4zV46cp | Hacker News Reader

Some people trivialise things in bad faith to score internet points on Hacker News, I guess. The status quo, per the article, was already using “deep learning” models to perform anomaly detection. We are already talking about complex, black-box systems.

The stated advantage was that they didn’t require deployment-specific training. You can just throw a pre-trained LLM at it. There is also a stated benefit: early-stage detection, without needing to pay for, or wait for, training a custom ML model. The article is quite open about the fact that the LLM approach doesn’t beat the state of the art in terms of accuracy.

It’s like you didn’t click the link.

View on HN

>early-stage detection, without needing to pay for, or wait for, training a custom ML model

This doesn't make much sense though, if you're in a hurry to get something working and don't have the data for training a custom model, you'd use the knowledge you have of the system to setup a typical statistical anomaly detector. That way you'd have the insight into the system behavior that you'd want early on, would be able to enforce all sorts of guarantees and would not have the weight and complexity of an LLM on the system's resources, rather than worrying about this LLM that has nowhere near the same info on the system.

Using an LLM for this would be like hiring a random person off the street to keep your network infrastructure working because you're in such a rush that you don't even want to document how your infra is setup, let alone wait to find someone who would know how to decipher it.

The metrics OP suggest increase with the transition from DL to LLMs (and OP had no immediate duty to discuss other metrics).

Colour me old-fashioned, but i am really sceptical about a zero-shot general-purpose LLM-based anomaly detector being able to outperform a trivial, not-obviously-wrong ML/statistical model calibrated on a small subset of the data that makes training extremely fast, in entirely new domains. And in anomaly detection, false negatives (and even false positives) can be very expensive.

The most promising role for LLMs in timeseries prediction is in extracting covariates from unstructured data, in the wind example, weather reports and geographically proximal social media posts.

You are skeptical about something that is explained quite thoroughly in the article. Did you read the article?

Edit: I meant they never claim to outperform SOTA domain-specific approaches. So, I don't know whose claim you are refuting.

> While LLMs could not beat state-of-the-art deep learning models at anomaly detection, they did perform as well as some other AI approaches.

I think one should also be sceptical about the current hype train

Funny how one of the the few comments actually discussing the content of the article gets downvoted while generic memes that have nothing to do with it get upvoted.

Reply on news.ycombinator.com