> Google's crawlers treat all 4xx errors, except 429, as if a valid robots.txt file didn't exist. This means that Google assumes that there are no crawl restrictions.
This is a better source than a random SEO dude with a channel full of AI-generated videos.
It's fairly common for there to be a very long and circuitous route between cause and effect in search, so a bug like this can sometimes be difficult to identify until people start making blog posts about it.
> I don't have a robots.txt right now. It hasn't been there in a long time. Google still shows two results when I search for files on my site though:
The source that he links to is another indian spam channel we've seen a thousand times on YouTube