Has anyone used AI to diagnose system logs? | Hacker News Reader

Has anyone used AI to diagnose system logs?

13 pointssans_souse17 comments

I did this with the panther file generated by running a trace on android and Gemini advanced seemed impressively capable and able to analyze the large file, and ouput a pretty technical breakdown of everything; pointing out culprit apps (often gApps) that are misfiring and causing latency issues etc. Was curious if anyone else has any experience and or opinions on this

View on HN

Yes

https://arxiv.org/pdf/2307.05950 [pdf]

https://www.cncf.io/blog/2024/04/12/streamlining-logs-with-o...

https://dl.acm.org/doi/10.1145/3597503.3639150

I haven't given it massive logs but I've given error logs to ChatGPT and it's helped me figure out subtle issues very nicely. Gives me a few ideas of what could be wrong, ways to diagnose them, and even commands to run to test or fix things. I've been shocked how well it works.

I worked on this project:

https://www.cs.dartmouth.edu/~dfk/research/aslam-kerf-news/a...

which had a machine learning component that automatically helped refine syslog SQL queries to find anomolies. More info here:

https://www.cs.dartmouth.edu/~dfk/research/project/kerf/inde...

Before LLM's.

The company [0] I work for gave a talk on this [1]. We're going a bit beyond analyzing logs because we have more contextual information about running software so we can compare different application state over time and infer whether it appears the application is experiencing an "incident".

0. https://noop.dev

1. https://www.youtube.com/watch?v=PxZx71xfIl8

My friend (former MSFT principal engineer) is actually developing this idea as a startup. He has a MVP. If interested, I can connect you two. Email in profile.

I tried a bit, but with not AGI. You can start to think how much would coast each request to your LLM. I stopped on that step.

Agree, logs are generated at an astounding rate usually. Using LLM to analyze those is implausible.

Host your own model?

What AGI ?

GPT4o. It's more useful and constructive, in general, than anyone who's gonna, with a slack jaw, beg the question of, "What AGI"

AGI doesn't have to do everything better than every human ever born to be generally useful and constructive.

If makes you feel better, think of it as dumbAGI - a generally capable AI that's not that smart.

GPT4o is a dumbAGI. You and I are dumbGIs.

Had a system where logs were watched to trigger various sound samples (bird chirps for firewall probes, etc). There was a pattern recognition component, when the entropy on that came up there was a "hmmmm?" sound.

Makes sense. I used to tell people that I didn't really need to read compiler output flying up my screen, but watch for the "shape" to change to know something was different. They looked at me like I was insane.

You're better at abstracting. I'd guess you'd also be relatively comfortable/competent with complex and/or higher order dimensions.

You were basically doing exactly what the crew of the Nebucadnezzer were doing when they were looking at the green Matrix columns scrolling down.

Your brain does the translating. I don't even see the code. All I see is exception, warning, errnospc.

I would never do this with a public AI system. If it’s running entirely on my own resources, that might be an interesting question to ask.

I struggle with dyslexia which makes it very hard to read through logs, so I often troubleshoot error logs, to summarize the logs and break them down by meaning.

I wouldn't ever do this in an automated way. However in person, it not only makes me solve problems maybe five times faster, but there are also many errors that I would give up on, which I can now solve easily.

Just the ability to turn a log into a human readable narrative does something remarkable. After I've done it once or twice, then my brain provides the narrative itself, and I can read the raw log without the need for assistance.

This suggests that not only does the AI assist me, but it assists my learning as well.

I also struggle with Man pages, in particular finding an option to do something that I need to do. I end up having to read through dozens of options getting more and more confused, sometimes skipping the option that I needed and not finding it. Now I can just dump the entire man page into an llm, describe what I need to do, and ask it to highlight the sections that I need to read. Or I can just ask it questions.

Reply on news.ycombinator.com