Hacker News Reader

> If you’re talking about SSL cert logs, that excludes a ton of web content.

I guess it depends on what you are looking for?

It's far less brute force than trying to randomly generate hostnames and then do lookups, and your solution also misses everything that is just random alphanumeric combinations, of which there are tons.

> And you haven’t shown me that you can even get all those for free anyway

Free? Maybe.

View on HN

Yeah, but because it's an LLM it makes very good guesses. It's not so random, it's seen URLs in training data.

One of the points of the experiment was to see how much of a shortcut using an LLM could give me, as the developer.

I've done other LLM search experiments where I basically just prompt to "predict 25 search results for this query" and it's surprisingly good. You can also run a check to verify the URLs to splice hallucinations out of the result, but yeah, LLM is already doing a lot of the work for search.

Reply on news.ycombinator.com