Geo Guesser identifies the location and seat number from an aerial shot | Hacker News Reader

Geo Guesser identifies the location and seat number from an aerial shot

twitter.com

296 pointshubraumhugo83 comments

This guy is accurate to a degree which I didn't think was even possible before seeing it. Here he is guessing locations based on just grass:

https://www.youtube.com/watch?v=iOCR73U8t6E

The scary part of this being possible is that I would think AI could do what he is doing even better than him, and at that point there will be a service where you can upload any photo and get it's location. This seems like it will be abused, and I dont see what can be done about it.

He has a few videos of him vs AI (that is specifically trained to play geoguessr) and it's quite impressive already just at that game-focused scale.

There's an episode in which he competes with an AI, but it turns out to be narrowing down locations using data most (if not all) humans couldn't. Imagine for example that the camera in a certain locale had some debris on it, but it's a unique shape and location on the lens. Or perhaps there's rain on the lens in various patterns. The AI neatly organizes these locations by their coordinates, then if it sees any other images with similar terrain and the same lens anomalies, it will guess with extremely high accuracy.

It left me wondering how effective it would be with current technology if it couldn't "cheat" in this way. I put cheat in quotations because in any situation where any metadata like this would be useful for location identification, it wouldn't matter how it worked. But, strictly using geological data, how would it perform?

> Imagine for example that the camera in a certain locale had some debris on it, but it's a unique shape and location on the lens.

I believe Facebook patented a method for using lens imperfections (dust, scratches, etc.) - see https://news.ycombinator.com/item?id=18835377

The patent office is going to change it so if a generic neural network put to a task learns the same technique as a human patent, the patent fails the ordinarily skilled in the art test.

Human geoguessr players use cheats like like less defects or other accidents with the captures.

You’re right, I totally neglected to point that out. I more so meant the AI is far better and doing this than humans are, and it’s wrong to say most/all humans can’t do it. We just can’t do it as well or as thoroughly as the AI can.

Rainbolt also uses these tricks wherever possible. Eg pro geoguesser players know how high the camera angle is on street view in each country, and which countries the edge of the Google car is visible on shot.

Definitely, that’s worth mentioning and I was arguably wrong to say most or all humans couldn’t do it. Many of the things the AI did really are things humans do. I think what I meant to say is that the depth and breadth in which the AI can do it is superhuman. Like, tiny bits of dust on the lens a person couldn’t really see become clearly and reliably identifiable features of locales. We can use aspect, seams, and other more obvious features of camera images in specific areas, but the AI can go quite a lot further using minute and sometimes almost imperceptible details.

This reminds me of the Tank neural net urban legend:

https://gwern.net/tank

I'm surprised at people who can't imagine what's theoretically possible. I always assumed someone could identify my location from a photo if they tried hard enough, even many decades ago before I'd heard of geoguessing or AI. It's just obvious that there's usually enough information in a photo to do that. Even for a grassy field.

There's a simple solution - don't publish your private information. That's how to use the internet 101. If you already did, then maybe it wasn't a very important secret anyway.

> The scary part of this being possible is that I would think AI could do what he is doing even better than him

Not a given.

Grass is incredibly distinctive, I can usually pick out if a photo is taken in australia from the grass.

It was so (irrelevantly) annoying that in a recent forza game based in Australia that they got the grass so wrong and it was so obvious to me.

> This guy is accurate to a degree which I didn't think was even possible before seeing it.

People forget geobolt is a performer.

My former manager at work posted a photo out of the window of his condo. I correctly identified his building, floor, apartment number, and which room he was in when he took it.

He was impressed and slightly horrified.

I'm no rainbolt, but the are a few neat tricks you can do. In my case, I was able to identify two landmarks that overlapped, and used Google Earth to extrapolate a line from them to his window.

Still, nothing compares to that time 4chan identied the location of a terrorist training camp from a photo and gave it to the Russian military who promptly bombed it (https://imgur.io/N7DwWP1).

The Shia LaBeouf “He Will Not Divide Us” 4chan geo-location is my favorite. Never underestimate bored autistic shitposters lol.

The initial couple HWNDU challenges were solved through rather boring means. They tried geolocating based on stars, but what ended up giving away the solution was a tweet from a local diner waitress posting a pic with Shia. Once they narrowed it down to a small town, then it got a bit fun, with one anon driving around and honking, so they could determine the location of the flag based on the volume of the honking sound on the livestream. And the whole drone usage to "alter" the flag was pretty unexpected.

But one of the latter ones was kinda crazy. I am thinking about the one where Shia placed a flag inside some wooden cabin (in Finland iirc), and they managed to find it based on a pattern of the woodgrain[0].

0. https://www.dailydot.com/unclick/4chan-shia-labeouf-cabin/

I think about the indoor HWNDU flag every time I post pictures online that were taken in my house.

The one I recall was when Shia finally pointed it at the sky with no landmarks visible.

So they tracked jets flying overhead, and combined that with what birds and bugs they could hear. And that got them the location.

We are talking about the same one, and there imo was some wishful exaggeration in a lot of coverage regarding how it all actually went, probably for the sake of making it into a more interesting story. I was in those HWNDU threads at the time, so I clearly remember it, and sources that looked at it in more detail corroborate it.

Tracking jets flying jets along with the rest of the things you mention only helped with narrowing it down to somewhere in the viscinity of Tennessee[0]. After that, there was no progress whatsoever until that geotagged tweet containing a photo of Shia with a diner waitress (she was the one who tweeted it, as it was a typical "i took a photo with a celebrity i just met" tweet), which narrowed it down to a speicific small town in Tennessee. And only then all the fun stuff with driving around and honking occurred.

While tracking flying jets was a cool idea, it ended up being a dead-end approach, as it didn't help with narrowing it down beyond "Tennessee and adjacent areas". And the geotagged photo that eventually led to the flag location didn't rely on overhead jet paths or any of that, as the geotag was fairly sufficient.

0. http://wikimapia.org/36787915/He-Will-Not-Divide-Us-Flag-Sit...

Some stalker managed to doxx a Japanese singer based off of reflections in her eyes on a selfie and identify which house and floor she lived in by minute details in the background of photos and videos [1].

Absolutely wild.

[1] https://www.bbc.com/news/world-asia-50000234

From the reflection in her eyes he "just" identified a train station, from there he followed her to the building she lives in. From posted indoor videos - and probably his own outside photos - he tried to identify the windows of her apartment based on things like window curtains and the direction of the sunlight. The linked article does not say whether he succeeded, just that he admitted to trying this.

this is why women generally don’t post pictures to the Internet without a lot of thought behind it

I know someone who was doxxed because the pattern on bedsheets that were visible in the picture uniquely identified her hotel. The picture only had a tightly-cropped cat sitting on generic white linen.

I know a person who made a comment on Twitter about “I need a new pair of boots” three years ago. From that, her stalkers correctly inferred what city she lived in based on historical snowfall data, and were then able to use other context to get her fired from her job.

Presumably this person got themselves fired for stuff they said online.

Or by owning an Onlyfans account.

> Still, nothing compares to that time 4chan identied the location of a terrorist training camp from a photo and gave it to the Russian military who promptly bombed it (https://imgur.io/N7DwWP1).

Image is too blurry to tell thanks to imgur compression, but does this mean the US-backed Syrian opposition?

Here he is asking for help to figure out where an old photo was taken ("a follower sent this to me in december, she had lost her mom 5 years ago and wants to know where they took this")

https://twitter.com/georainbolt/status/1698888344451867039

I like that he cares more about that than keeping up appearances

Something I never see mentioned about these Geoguesser streamers: what if they are cheating? It would be trivial to set up location extraction the Street View http requests and show it on another monitor (or even overlay window hidden from stream capture) that the streamer can glance at.

I feel like you're trivialising triviality. You may be capable of doing that but it's not "trivial" by any stretch.

To be fair it is trivial compared to the skill displayed if there is no cheating involved.

Rainbolt has made a video with an eye tracker which would be difficult to fake (I think): https://www.youtube.com/watch?v=aD_rtwofvgU

In October there will be an in-person event: https://www.geoguessr.com/world-cup

edit: in this video he also includes a third person view of his monitor – https://www.youtube.com/watch?v=izarzOXjZp4

Some probably are cheating, but at least some are in fact this skilled.

its a game. the site intentionally picks photos with regional identifiers so you can get good at it

Ctrl+F: AI

Probably the biggest change AI has brought to my life as a developer is that every single thread turns into "what could AI do"/"AI can do this cheaper"/something something AI future. Much more impactful than any AI product.

Identifying a location like this is absolutely a skill that humans can perfect, and that should be celebrated. Going so far as to train yourself to recognize grass and little details like this is absolutely some kind of mental athleticism. There ought to be some kind of a tournament!

Nothing new there, there's a revolving door of frequently named technologies on here. Machine learning, blockchain, IoT, etc. Hype cycles come and go.

Revolving door? Is that a new Agile strategy?

It’s an ethereum tumbler

Back when I used to do CTFs a common challenge was to set up a fake Instagram account and have the flag be the coordinates for the location in the image. Good skill to have, and to know how to protect yourself when you share images online.

This reminds me of when the FBI caught a child kidnapper using a blurred street sign.

Law Enforcement and Search & Rescue would be smart to hire these people

They do. Someone I know had a bit of fame for helping find a lost hiker from a photo.

https://twitter.com/ai6yrham/status/1400806794667102208

There was a pedophile that blurred a photo of himself using the swirl feature in some graphic editing software. Police just simply 'unswirled' the photo to get a clear image of his face and arrest him.

I have a large backlog of photos taken before it was common for cameras and phones to automatically embed GPS coordinates in the files. It would be nice to geotag them. I would pay for an AI service that could automate this with >85% accuracy.

It seems like the answer is to make a game out of it, and invite geo-guessers to participate.. although, it'd be hard to score since you don't know what the correct answer is. Maybe make it a crowd-source webapp..

Impressive, but I don't see the seat number?

It's on this reply: https://twitter.com/georainbolt/status/1698554653854826984

(For those confused: when not logged in, Twitter doesn't show replies anymore, only a single tweet, like if they didn't exist at all; it doesn't show almost any tweets on anyone's timeline also, except a few old random ones. Thanks Elon.)

The seat number is imo the least impressive part. The wing gives it away.

More info on the actual flight and seat number in his video: https://www.instagram.com/p/CwybO1tvI0O/

The seatmap: https://seatguru.com/airlines/Southwest_Airlines/Southwest_A...

He wrote 17F on twitter, and said 18F in the video. Which is it? :)

That's your question, that he might have been off 30-40inches or whatever is the distance between two economy rows in a737? :))

Not that impressive. The post has flight # (you can lookup seat map, direction of flight), and wing photo.

Ask anyone who’s been in consulting (like big4) and they’ll all have a good chance of getting it right.

Big4 is accounting

Deloitte, Ernst & Young, KPMG, and PwC would all have significant time around airlines and aircraft manufacturers, all of their various projects, diagrams, data, etc. For lots of projects that have nothing to do with accounting. And also a lot of miles sitting in those aircraft.

It's because you need to be logged into Twitter to see more than the single linked tweet.

I don't care about who runs the web site, this change in behavior is a horrible experience that's different than any other link on HN.

I totally agree with you. I deleted my X account and only find myself on the site when I click a link without noticing the URL. I don't miss it, but it would be nice to have more context for what people are discussing.

I read the tweet, realize there’s more, realize I can’t read it, and feel sad for a second because it was probably good info. Twitter is such a negative place that even FOMO isn’t strong enough to draw me back in.

I feel like I’m missing on a lot from my local community by rejecting FB, Twitter, etc, but it is bad for me, and I feel much better without it in my life.

Honestly this just makes me want to give GeoGuessr a go again. Back when, I jumped in and the first shot I saw was like "What? No-one could guess where this is." Now I know someone can, I realise the game is playable.

Why are people still posting direct links to X here?

To link to the original, and best, source.

There's a wing in photo and flight number already given. Figuring out the seat is the easier part.

Would love to see a modern AI trained on geo-guessing compete with a human.

https://www.youtube.com/watch?v=ts5lPDV--cU

this guy is such a meme. i wonder if anyone is out there keeping track of his hit rate, because social media only serves up his biggest wins. i'd be very curious how his hit rate improved over time.

Try watching him on YouTube. He’s definitely on another level from other high level players.

I was under the impression that he's a high level player but far from the top of the leaderboard. Although he's currently the best GeoGuessr content creator and makes the most interesting videos in GeoGuessr adjacent topics such as finding locations from tweets and other viral videos.

Geowizard will always have my heart on those. It's true he's not as extremely good as the current leaderboard, but he's also good at these adjacent topics such as finding the location of historical photos, locating fan-submitted challenge selfies etc. And I mean, no one would come up with the scheme of crossing countries in a straight line without a pretty passionate love of geography.

You can watch him livestream some incredible shit. He's absolutely the real deal. Yes, he's big on social media, but every niche hobby like this has a standout who's great on media (Hikaru comes to mind for chess) that does great things for promoting the community.

I think the majority of Hikaru's popularity comes from how dominant he is at chess, particularly faster time formats like blitz/bullet.

Seems to me he still commonly demonstrates a level of sportsmanship such that the online chess "Sportsmanship Award" is named after him.

Oddly enough that os also what georainbolt is known for in the community!

It warms my heart whenever people are rewarded for being 1. Amazingly skilled at a thing, 2. Kind-hearted, and 3. Good at video production. No need for clickbait, outrage, trash talking, etc. Just quality content of someone doing something well and being nice about it.

Surely if you don't hit, you can keep looking. Unless it's actually impossible for some reason.

thislocationdoesnotexist.com seems like it'll be a thing created just to mess with these people. why? because someone can.

Nothing stopping a smart person recording every sparce tree and building on earth at home and automating this.

Except time, and maybe Google Earth blocking the scraping

Here's a challenge though: For every correct guess, how many wrong ones are there? From this person or someone else.

Because all I'm seeing is people who are correct, always, which is suspicious. It's not exceptional unless it's actually uncommon to be so right all the time.

I think, that once you have a guess, it's easy to verify it before posting it online

They also do random timed challenges, you may check these if you want numbers.

Just watch the live feeds then to find out

Reply on news.ycombinator.com