I think the main problem with the google charts API is just that, it's not a library that is self contained. You need access to google's servers. Second, if you have sensitive data, you rather not send it to google.
The only problem I've run into is an issue with URLs being too long when trying to chart large amounts of data.
Google Visualizations API is also very good and (I think) avoids the problem of sending data to Google, and if locally cached, shouldn't require access to Google servers. That said, if you're running an online app, you should have access to the Google Servers, especially if you're already using their copy of jQuery for performance reasons.