The web UI for people using search may be obsolete, but search is hot, all AIs need it, both web and local. It's because models don't have recent information in them and are unable to reliably quote from memory.
I see search engines as a dripfeed from a firehose, not some magical thing that's going to get me the 100% correct 100% accurate result.
Humans are the most prolific liars; I could never trust search results anyway since Google may find something that looks right but the author may be heavily biased, uninformed and all manner of other things anyways.
Crawling the web has a huge moat because a huge number of sites have blocked 'abusive' crawlers except Google and possibly Bing.
For example just try to crawl sites like Reddit and see how long before you're blocked and get a "please pay us for our data" message.
95% of our load is from crawlers, so we have to pick who to serve.
If they want our data all they need to do is offer a way for us to send it, we're happy to increase exposure and shopping aggregation site updates are our second highest priority task after price and availability updates.