Curious to see if ignoring robots.txt, which has almost miraculously held up for decades, could eventually lead to a lawsuit.