* Google's data seems to be not as fresh as Kayak or Hipmunk's is. There have been multiple times when I've tried to book a flight via GF only to be told that there were no seats left when I arrive on the airline's page. I suspect part of the greatly improved speed comes at the cost of aggressive caching that runs a risk of things being out of date.
* Google's data is all over the map on pricing. Very frequently the airline's price will be much cheaper than GF. And more than once (especially on United or American Airlines) I've noticed that if I book directly through an airline over a VPN, I save several hundred bucks versus if I click through from Google Flights or Kayak. Conversely, sometimes GF is less expensive and when I land on the airline's page it's more expensive.
I think the best thing to use it for is route discoverability: which kind of planes and which airlines fly to which locations, on which days at which times? It's also great for doing complex open-jaw routes or hidden-city fares -- and since those routes don't change very often, the data is more likely to be accurate.