An interesting development is AI bots that can act as human agents and visit a site and extract information. I imagine these are only needed when there are barriers to use all other solutions, otherwise it’s a bit overlillt