.. as the webmaster implemented something that they might thought has an impact (false sense of impact), but has zero
so net gain negative
i consider such lists harmful - a good website is one that supports the goal of the website providers and its desired users (some of these users might be bots)
a bad website is a website that does everything for everyone just because
When I was younger I would have though the same. Now that I have more humility and less working memory, I think differently.
You won't find generic lists that are well suited to your case, and you certainly won't find any flawless one. If you don't know the details about one of those items, you either go with "no" or learn them. But there is a lot of value on getting a list you can look at and discover something that you forgot.
True, but it serves a other purpose, especially when the website is offering developer-oriented services. It's a single link you can give your AI agent and ask to "read this, understand it does, implement it".
Sure, you could just point it at docs.<service>.com but there might be bot protection, authentication, JS-heavy content etc.
So i feel llms.txt still has a purpose.