That in theory means that if your details are on a website, you've already given consent for it to be shared with Google.
Does that happen in reality? Most of the time I doubt it. But it pushes the liability onto the website owner.
UPDATE: I suppose if you specifically allow the Googlebot via robots.txt, then in that case they could probably argue that you gave them permission to access the site.
This is worse than shrinkwrap agreements and TOS banners at the bottom of a website, since for those you could at least argue that the person who opened the box/visited the site saw the agreement. But in this case, there is absolutely nothing to indicate that the person who created the website even knew that Google as an entity existed.
In theory you're right. In practice, just because something responds publicly does not mean it's public.
If someone left the vault door open on an ATM it doesn't mean you can just have the money, or even take pictures of the insides.
Obviously, in IT, people should strive to prevent leaking information, but that's obviously going to be an issue for a great long while.
If anything, that's more to the discussion of giving social security numbers to inept IT departments.
So yes, you do allow them to visit your website.