It's possible that your site's IP was used previously for some other sites and it was indexed as http://IP/... now google will retry to fetch the urls for a time.

Maybe you have someone's domain resolving to your IP. Seems to me that nginx is resolving the IP to the default site in the config.