Hacker News Reader

Comment by bbu | Hacker News Reader

This is pretty cool! However, these two samples are very simple to solve. I'd love an "AI" to find root causes for problems that are not obvious. Just throw the whole log collection at it and let it solve all the issues. One can dream ;)

View on HN

Yeah, my experience so far has been that if I just pile a bunch of logs in there, if it's not given salient lines, the language model tends to either rat-hole on some particular detail that's irrelevant, or else construct a non-factual narrative. But, when Zebrium did the picking of these lines autonomously and GPT-3 summarized them, we see a meaningful summary. Having said that, I would also like to get GPT-3 more and more savvy with larger and larger log-based prompts, and I'm hoping it's possible with some tweaks to the prompt and some finetuning. I'll keep the blog posted as we do more experiments.

this is really cool!

GPT-3-stackoverflow

But seriously, was Stackoverflow part of the training data used to train GPT-3? Would definitely be an interesting fine tuning experiment

From what I've read, the answer is "yes", stackoverflow was crawled. EDIT: I looked and stackoverflow is included in the Common Crawl dataset, which is one of the datasets on which GPT-3 was trained. Having said that, it's not clear to me the degree of coverage of that domain guaranteed by that crawl... looks pretty comprehensive, though. http://index.commoncrawl.org/CC-MAIN-2020-24-index?url=*.sta...

The training data was so large that I don’t think a single epoch was used. I.e. it’s not a guarantee that any specific site is known by GPT-3 just because it’s in the data.

But, with the nature of random samples, it’s probably likely that it saw at least a little bit of SO.

Reply on news.ycombinator.com