I scanned 1M domains and found the web's AI instruction layer

open source MIT License python crawler https://github.com/DialtoneApp/crawler