Just this morning I had a worrying result when I googled "UV index <location>" and it confidently told me the current UV index was zero.
It was not, and at least because it was obviously wrong I clicked through to a page with the real UV index.
But had it said "The UV index is 1.7" I might have believed it, but it'd still have been wrong, potentially dangerously so.
Now to be clear, live data is precisely the sort of thing I expect it to get wrong, and I never intended to trigger let alone read an "AI summary", but it's still worrying for those less inclined to dismiss it.