StreetLearn: Learning to navigate in cities without a map | Hacker News Reader

StreetLearn: Learning to navigate in cities without a map

sites.google.com

53 pointsespadrine11 comments

The agent learns to navigate entire cities by exploring them on the ground (it sees only Google Street View photos), without any map or location data.

The human analogy would be to ask you to learn to navigate a new city (whose language you don't understand) without ever using a map or location data, simply by driving around and looking out of the car. You will drive around and, as you see different street scenes, you will gradually build a "mental model of the city" that allows you to navigate it.

That's essentially what the agent does.

Once trained, when the agent is asked to go to a particular location, it knows how to get there, despite never having seen a map of the city.

Impressive.

Oh, I thought this was going to be a technique for humans to find their way. We need some other word for "learning" that specifies "training a computer model".

My two tricks: satellite dishes point south†, more or less, and Christian churches are usually built with the altar pointing east.

---

†In the northern hemisphere.

Satellite dishes point south in the northern hemisphere, right? Since they orbit over the equator.

I've never found this practically useful, because the one city where I've wandered a lot and saw a lot of satellite dishes is Philadelphia, which has a grid layout. But I do recall noticing it.

Yes, of course! Sorry for that. Being European I sometimes envy not only those neat grid layouts, but also the numbered streets, because from the numbering you can infer the position of the target. But "Saint Joseph's street" could be at the next corner or on the other side of town, you cannot know...

Moss grows on the north side of satellite dishes. In the northern hemisphere.

Remember, you can also go into the nearest pub and flush a toilet to determine which hemisphere you're in.

So, is this picking up on large landmarks which are visible in StreetView? Sun position in imagery? In Manhattan can it see a sign reading W 53rd St and know whether to go north or south to its destination? The paper suggests that there are learned landmarks, but doesn't talk about street signs at all. As a New Yorker, that would be the first method I would use to reach a destination.

It's a neural network so chances are, we'll never know how it came up with it's decision. For CNNs we can look at the output of each layers to get a hint but the more diverse the architecture is (such as in this example) the harder that becomes.

It went backwards all the way in London. I wonder if it didn't know which side of the road it should have been on... Might have helped going through some one way streets.

That's just due to Google maps street view - the original images happened to be recorded in the other direction. You'd have the same 50/50 chance on any two way road.

Reply on news.ycombinator.com