Is there a reason Google doesn't derank these types of sites? I've seen the first few results all be these types of articles for some search queries. I'm sure it wouldn't be that hard to detect these sites to flag them for review.
They probably don't mind search results turning to shit, might make their 'AI mode' more palatable to users
Quite the opposite, Google actively featured this hallucination. Hilarious.
But what can one do now? It's no longer inaccurate :-)