It's not just Google. Other GPS maps will send you to my house on a road that hasn't really been passable for 30 or 40 years - if you make the mistake of not telling it to avoid dirt roads. But, if you tell it to stick to paved roads, it'll have you park about 2 km away.
I have no idea how to fix this, except to suggest that they update the maps that the GPS relies on. It's such a remote area that it's not a priority for them. Every year we get a few people who think that GPS can't be wrong. Maybe if one of 'em dies and someone sues they'll get around to updating it.