Facebook machine learning technology improves; Redditors alaramed.
reddit.com
reddit.com
He told me that when he went to different divisions in each company, they all had org/structure charts somewhere around the office. And in every one of them, their section was in the middle and everybody else was serviced by them. No matter what you did, you were the center of the universe to you.
Likewise, I've had experience building and working on many computer systems for businesses. Each of them, if successful, wants to handle not just their thing but everything for their customers. No matter what they did, they wanted to be the center of the universe for their customers.
You can laugh at these examples, and we all know most software companies have much great ambition than they do traction, but Facebook and Google are actually doing this. They are becoming the center of the information universe to their customers.
This is not a good thing.
until someone provides a POC that states the opposite it's just YOUR data that YOU freely give to fb. no magic behind, just FUD.
responded to top post for karmawhoring. http://news.ycombinator.com/item?id=3377267
Like many old farts, I imagine, I see your post in terms of the thick-client/thin-client battles of years past. I don't think FB will ever see it in their interests to completely close their garden, which would really be remaking the mistakes of the past, but stranger things have happened.
> Loyalty, not lock-in. We firmly believe you control your data, so we have a team of engineers whose only goal is to help you take your information with you. [2]
It’s absolutely terrific that they have something like this. Many others had to be dragged there kicking and screaming.
However, this is just not relevant to the topic at hand. This is “your data”, the data that you deliberately created, rather than revealed accidentally. The point of the project is to avoid vendor lock-in, rather than to protect your privacy.
If you’re EU resident, you should be able to just write to them and they have to reveal everything they have on you. And that list will be entirely different from this.
[1] http://www.dataliberation.org/
[2] http://googleblog.blogspot.com/2011/06/supporting-choice-ens...
Why isn't it?
Then, if the company decides to sell your information, lose it, etc., it's too easy, and you're left empty-handed.
However, I think that if that 1 service makes it possible for you to backup all your information and allows you to remove all the information from their service if you want to, then it shouldn't be a problem... but I might be underplaying their argument...
Something like Google's Data Liberation Front? http://www.dataliberation.org/
I put all my eggs in one basket every time I take the family to the grocery store. It's unclear to me why storing all my information with Google is any worse than piling my wife and kids into the Odyssey.
Google's goals and mine aren't aligned at all, except instrumentally. So Google cares about my privacy up to and until it costs them less to not care than to care.
I'm fully in the Google camp, but let's be honest about the fact that we've made a deal with some supernatural being. We just hope it turns out not to be the devil.
Just wanted to balance the “Facebook evil / Google not evil” diatribe, as when discussing these topics only Google is ever mentioned.
Google gives you the email addresses because you provided them with that data, it's yours to begin with. Facebook will give you the email addresses, but only for the users that allowed it to share that data with you.
Will Google give you the email addresses of your Google+ contacts, if they decided not to share their address with you? I doubt it, and rightly so.
“Data Liberation” means that you own your data, not that using some social network gives you the right to be able to communicate with people outside of that service. If you want to contact them outside of Facebook, ask them their email. It's your problem, not Facebook's.
Your argument is like saying that Google should give me my Gmail contacts' phone numbers, so I can contact them without using email.
Say you use Google Apps For Domains. Thinking it is safer because you own your own domain. Arguments that you don't really own that domain aside, if google dies, you won't be able to transfer that domain away from google.
Email is the backbone of an approval system for just about everything.
I think you can pretty safely put all your eggs in one basket, as long as you take one egg out, that being your email access. At the very least, forward all inbound emails to some other service so you have access to those in an emergency.
Ideally, we would all have our own mini email servers that we control and manage. However, managing an email server is perhaps one of the harder chunks of tech to manage. From anti-spam to just configuring all the pieces, it is much more than most are willing to do.
If they can make guesses this accurate about your photos, imagine they guesses they're making behind the scenes about your life, your personality and your innermost thoughts.
If there was a page on fb where it showed all the inferences they had made about you, sexuality, income, religion, philosophical viewpoints, mental health, etc. then people would run screaming. Of course, a lot of people have already told fb that info voluntarily, and that's why its possible to guess it for everyone else.
Also potentially terrifying: a facebook fortune telling engine. I bet they can predict your future with frightening accuracy, or they will be able to after another decade or so of data anyway.
I'd like to fantasize that there could be a way to use a social network like Facebook with a tool like Ordo -- that is, most the data (photos, in this case) you upload to FB is encrypted, and only people who you've approved can decrypt it. Using steganography this is probably already possible with FB, probably not practical, though.
Even if you were to cancel your account you'd turn up in photos of parties on other people's accounts and they'd follow pretty well you after you'd left too. With their dodgy history, Zuckerberg's contempt for his own users as evidenced by his quotes, the idea that this won't be abused is laughable. Given national security letters (NSL) it's almost certain the intelligence community has access. Hell you could hit a mid-level admin at one of their data-centers with a NSL and the top level management and legal wouldn't even be aware that there are outside entities with a connection to their database. That's to say nothing of the extra-legal capacities of these people.
While people like to think that it's more or less anonymized clusters and demographics that are being passed to advertisers to target ads better - not a terribly scary thought, just more relevant ads - I think we need to be far more cautious.
How do you go about convincing the "I have nothing to hide" majority of the population that giving away such vast quantities of information is not in their best interests? I'm thinking of friends, family members, etc. who honestly don't care whether the TSA sees them naked at the airport, or how much the other agencies know about them. Where does such extreme deference to authority come from in the first place?
Gilles Deleuze said it comes from a desire to be led.
Continuing and possibly gruesome examples where such data mining ruins lives.
I spoke with a recruiter who prided himself on their HR department's willingness to deny entry based on FB profile (unsure if was public or obtained through some "secret friends" policy, or through a backdoor).
Furthermore, it's becoming almost a certainty that some foolish judge will set precedent somewhere that essentially makes FB account data public (or forces publicity of formerly private data).
But privacy is not just about hiding bad things. A multitude of innocuous facts about many people, collected in a database, are themselves a powerful tool. The data can be mined by governments to look for 'suspicious' behavior, by insurance companies to deny insurance, by criminals to commit fraud, etc.
I think the basic problem is that violations of privacy give power in the hands governments and corporations, and these entities tend to abuse whatever power you give them.
[1] http://chronicle.com/article/Why-Privacy-Matters-Even-if/127...
If you are a home owner and this is true, you could possibly be identified -- say for instance posting from home 75% of the time and cross-referencing with certain demographic statistics collected from your browsing history and property records. Would that be legal?
(The problem, of course, is that a "distant future" today is 10-20 years... Probably even in my lifetime.)
Also they are a fortune telling machine. That's a good pun actually. They are hoping to make a fortune by telling you what ads you'll click.
Some of them where right, but more than half of them was wrong (I am not, for the record, interested in Paleontology).
See "Your categories"
Edit - Sorry, I copied the link from the Google results page, it acutally shows up as adspreferences in the ad, but ads/preferences below. Kind of strange.
In all likelihood, there's no magic. Its just comparing 'dumb' manual album labels with places pages and trying to match them up.
I've had similar experiences with the location-suggestion feature where I was totally bewildered by how it was getting the data to recognize the locations. As it turns out, all of the albums they've done this for were pre-location tagging and so I'd manually put in a location (like 'Bowery Ballroom'). So there wasn't any particular magic in how they seem to be doing it. This would also explain one of the other comments on here about the location suggestions being England, Arkansas (if she just labeled the album England and the location suggester goofed). I also had it goof when I had an album labeled "Rhode Island and Massachusetts" and it tried to suggest a real estate agency with that in its name.
Henry F. Bridge, Product Manager, Facebook
Photo albums on Facebook have long had a text input field for location, but until recently, there was no way to put in structured data in this field (like a Facebook page). We added that ability earlier this year, and the "add a location" feature just performs a search on whatever text the album owner put in originally (ranked by place popularity etc) and suggests the first result as the location of the album.
Scary stuff.
Facebook almost certainly has more photo information than TinEye or Flickr, and of indoor environments probably more than Google (which has reverse image search too). Across any given bar or hospital Facebook would have maybe 5-10 other people with albums tagged with the name/gps/check-in. They'd only need one other album though.
SIFT more or less turns every image into a bag-of-words. Your single photo, even at different angles, is going to have a heavy match with photos they have. If you upload a whole album they are going to have tons of matches and they can be more or less certain of the location. To say nothing of adding even the most basic geoip-to-city lookups that would narrow you down to at least five cities that you and your social network inhabit. But the extra information they have is besides the point, SIFT is enough; hospital rooms look alike to us, to SIFT they don't.
Then you have the data sitting around for mapping the insides of all buildings like Google has it for street view. Sure your Asimo-style robot butler/"something really BIG" will be that more efficient with internal mappings of most public spaces when you release it in 2030, but the convenience is sufficient. Remember that Facebook beat Myspace more or less on interface (combined with a few other factors like exclusivity), they aren't going to let Google or some other competitor get ahead of them by having an interface convenience edge of 5-10% which might cause a Myspace-to-Facebook style exodus. Kings who have committed regicide on their predecessor are all too aware of how they got into power.
Pictures leak a ton of side channel information outside of their subject matter.
There's probably hundreds of photos of babies being taken with GPS from that hosptial.
FB image recognition probably thinks all these babies are the same "person".
Then a separate system comes along and takes photos that do not have location info and tries to match them up with ones that do. It finds a match with the "hospital baby" and then asks for verification. Person says YES and it adds that baby to the "hospital baby" pool as well.
"I just had a baby" - updated from hospital (Upload photo with timestamp near that status time) Match the two
Well, to SIFT they'll all look different. Except for the hospital room in Portland, OR and the hospital room in Portland, ME that each happen to have a sign with "Portland Hospital" visibile in the picture. While I'm alarmed about the potential of computer vision to compromise my privacy, I have yet to see anything in actual use even be competent, let alone alarming.
The problem is that single SIFT features are not very distinctive, and so you need many of them in common between two images to get reliable matches. So you essentially need images taken from very similar locations. You can ameliorate this requirement slightly by using fancy tricks in how you aggregate SIFT features together, but the fundamental constraint remains.
This precondition is satisfied at popular landmarks, since there is a good chance that someone's taken a photo from the same location you're standing at, but in general, this condition is not that easy to meet, and hence the poor accuracy of current approaches.
Finally, SIFT is great only for roughly-2d, distinctively textured areas. However, a lot (maybe most?) of the world does NOT fall into this category -- many things are either not textured, not distinctive, or are 3d. Something other than local descriptors (of which SIFT is the best known example) will be needed to understand these kind of scenes.
If you're curious about this line of research, a good place to start is the IM2GPS paper, which was the first major work (that I'm aware of) to look at this problem: http://graphics.cs.cmu.edu/projects/im2gps/
I took some pics on a regular digital camera (no GPS) in Indore. I uploaded them from New Delhi a few weeks later. And now FB is asking me "Were these pics taken in Indore?". Crazy shit.
Update - I dug through my FB updates. Just before leaving for airport, I updated my status to "Off to Indore" and after coming back to New Delhi, I had some status updates about my office and a local park. Facebook is probably using the the timestamps from image and relating it to locations using some heuristics like status updates, IP addresses, image recognition etc.
doesnt take much to throw out a guess based on a single GPS location + timestamp. Even if the guess is only right 5% of the time it is still a profitable guess to make.
According to a redditor,
"
- It's not IP address. Facebook successfully identified a number of specific locations (bars, theaters, etc) even though I had uploaded the photos from my home
- It's not geo-tagging. All of my photos were taken with a camera that does not geo-tag (Nikon d700).
- It's not contextual tagging. There were no people tagged in the photos, no comments in a lot of them, no words or phrases or names in the captions that could have given clues
- It's not image recognition. One set of photos was taken at Cafe du Nord in SF, CA and every single shot was of the performer onstage, with no identifying characteristics or clues to be had.
"
I would really like to know as this is very interesting and none of the reddit comments (as of now, 12 hours after submission) really answer this question. What technology or methods are they using to suggest (accurate?) locations where pictures have been taken from?
Even more strangely, I have never used my mobile phone with Facebook, but when I uploaded a photo just now of a place from my childhood to which I haven't ever been since using Facebook, Facebook correctly suggested the location.
What the heck?!
From an image recognition standpoint, if anyone else was at the event and took similar sets of photos which they then tagged, that could also be used.
It's a picture of a baby at a hospital.
There are probably hundreds of baby photos being taken per day at that hospital being posted to FB. Some have GPS and some don't.
The images that do have GPS, get recognized and lumped together as the same "entity" since they all look the same.
FB then looks up where she lives, then looks for any photos that don't have a location (like her baby photo), then tries to match it with any photos within a 50 mi proximity to her hometown. Bingo finds 90% match on the baby pic in the hospital, and asks user for verification. She says YES, then this pic gets lumped together with all the other entities. FB also asks more often since it's "right".
If so, we're in trouble...
Redditor has facebook app installed on their smartphone (or just uses the website), sets status to "OMG wife is going into labour, at the hospital now". Facebook now knows roughly where redditor was at the specified time based on the ip, they can narrow this down further by looking at keywords in the status message and check it against a list of addresses in the local area and select the best match.
When the redditor comes to upload their photos days or weeks later, facebook just checks the photo timestamp against the user's location +/- X hours and makes a guess at where the photos were taken.
That's the interesting question here.
If they can, then they must be doing image recognition in some way. Perhaps someone who has more time / energy than me could try this by creating a new account and uploading some photos of recognisable objects (but without exif geo data).
No real trick to that; if the user signs in from an IP leased to a Christchurch, NZ ISP on March 5th, 2007, then it's a safe bet that any photos taken that day were taken near Christchurch (and you get date taken from the EXIF data of virtually every camera on the planet).
When you uploaded the photos: [hh:mm dd/mm/yyyy, photo] + above data => [photo, location]
Or, you used facebook at some point while in that city, facebook recognised that your photo was of a baby just born, by comparing it with the millions of other photos tagged with "OMG MY BABY IS BEING BORN!," it knows that babies are born in hospitals, and it knows that there is only one hospital in that city, so it guesses that the photo was taken in that hospital.
Anyway; in 2005 I went on a skiing trip to Chamonix. Pics were uploaded about a month later, no Facebook activity at the time (or while I was there). The other day FB asked me if they had been taken in Chamonix. So... something else used there at the very least.
Regarding your Chamonix images, perhaps your friends uploaded photos taken at roughly the same time, with geo-tags? Perhaps you are tagged in some of them? That would make for pretty simple logic:
- Facebook knows that you were at a certain place a certain time through the geo-tagging of those photos, as your user is tagged in one of them
- You upload photos taken at the same time as the photos you were tagged in
Conclusion: You must have taken those photos where the photos you were tagged in were taken.
Nope. Family holiday. Of those there I am the only one with photos on Facebook - and in fact at the time was the only family member with a Facebook account :)
I am intrigued because if it is image recognition there isn't much for them to have gone on. But I am stumped for what else they could have used.
My creepy moment came when It correctly pointed out the exact location of this photo:
http://www.flickr.com/photos/40127665@N03/4788700749/in/set-...
I was in the middle of Costa Rica. For fear of roaming charges my phone was not on, and I never made a status update. The photo was taken on a D40 at the time, so no location exif data, and frankly the the picture is kind of generic.
My best guess is it used the other photos in the album to gain contextual information. For example this photo was in the same album:
http://www.flickr.com/photos/40127665@N03/4789329722/in/set-...
this, to me, would be extremely easy to recognize.
I'm yet to be convinced that this feature requires actually analysing image content at all. GeoIP and time stamps alone can provide a huge amount of context before you even start on things like mobile clients providing location data.
Kidding! If it was that good, I really would be shocked.
I haven't checked to see if the individual photos are correct. However, at the time I didn't have a smart phone and I'm pretty sure my camera wouldn't provide much useful info.
UPDATE: I take it back: fb thinks I was at a football camp called St Catharines and Niagara! http://www.facebook.com/pages/St-Catharines-and-Niagara/1821...
I think that shows that the title is the most important indicator to fb.
"You know what this means:
Time to rewrite your EXIF info and location bomb the hell out of popular attractions. Eiffel Tower in Paris? Nope, it's in Iowa ..." [1]
Possible? Google bomb with a twist?
1: http://www.reddit.com/r/WTF/comments/nkktm/facebook_is_reall...
http://www.reddit.com/r/WTF/comments/nkktm/facebook_is_reall...
""" I am an engineer at Facebook working on the Locations team. We use the text entered in the location field of the album by the album owner and match it to the best guess for an existing Facebook Page using text matching. The popularity of the place is also used to rank the suggested places. """
http://www.reddit.com/r/WTF/comments/nkktm/facebook_is_reall...
(I work at Facebook, not on locations/photos.)
I've uploaded 4 screenshot-ed landscape (including one building) pics from flicker with absolutely no metadata. at least 2 of them should be very recognizable. the only thing that fb suggested was the single check-in that i had, minimum of 500km off.
i'm pretty sure that there is only minimal (if any) intelligence on image recognition and that they guess locations by information that "leak" from you and your hardware.
I have Facebook suggested a few albums today (2 hours ago). Some of them are from 2+ years ago, others are recent ones. The suggestion is quite accurate (89%, 8 out of 9 albums). With the incorrect album, Facebook suggested another city with the same name but from another country!
So Facebook probably just combine as much data as possible and when the matching rate is larger than some threshold, it will temporary tag the album and confirm with users. Data may include:
1. Album info (yeah, most of my album includes the place in it's name or description)
2. Comments group. 3 of my albums have a large number of comments, they are all from my highschool friends (I grouped them all in the same group) so it's quite accurate I guess. Also, if Facebook use the information in their smart lists, that makes sense too.
3. EXIF data. Some people says Facebook can read the data and cross-check the date with its database of check-in. This doesn't happen to me but I think it's possible. And of course, if the EXIF has geo-tagged, that info can be picked up.
Thought?
Maybe some neat work with GIS enabled this, but I don't see anything that strongly indicates machine learning.
http://www.reddit.com/r/WTF/comments/nkktm/facebook_is_reall...
It would be easy to train this system by displaying a "guess" (a friendship recommendation for person A) to persons B and C. If person B or C show interest in the recommendation then perhaps a relationship exists. Guesses could also be formed by comparing keywords/metadata found on profiles in two different social circles.
The reason I mentioned LinkedIn is that I think they do a better job of recommending/guessing who your acquaintances are than Facebook.
Facebook usually suggests people I have met but do not want to be "friends" with.
There is an album now with random photos, with the location "Plekke" (which is afrikaans for "places"). They are suggesting "Plekke, San Juan, Puerto Rico". So, currently I wouldn't suggest machine learning, although it is not entirely dismissible. They have the largest photo db in the world, they could eventually learn and then seed it with stuff like checkins, status updates, etc.
EDIT: I have another album called "Kuala Lumpur and Thailand", which have pretty iconic pictures in it (Wat Pho, KL Towers, Batu Caves, etc) and they haven't asked me about that album at all.
I have a Nokia phone with no GPS. Many programs uses the cell tower info added by the phone to determine where the photo was taken.
Upload the photo using your facebook account. Check whether they get the location right. If yes, the mystery continues. If no, but the location is somewhere other than where you updated your status/checked in from, the mystery deepens. Else, they're pairing the times together.
Given that most of my pictures were taken with cameras with no geo identification mechanism, I can only conclude that they identify the location based on any comments and similar photos available in _my_ network.
But I think I finally figured it out in my case. I had used Picasa to upload the pics years ago. In Picasa, I had entered a caption with enough detail for Facebook to guess the location. The caption had never been uploaded (i.e. set as the photo's caption on Facebook), but I'm guessing they did somehow capture that info on upload.
Why do they think people would generally be interested in clicking "Yes" on these suggestions? That they are helpfully filling in the blanks on photos I "forgot" to location-tag?
Facebook wants its users to catalog their lives. For them, this data is marketing gold. For end users, it means their content (photos, status updates, life events, etc) are categorized more thoroughly and in a more meaningful fashion (geographically, chronologically, etc).
I think that real world places are an important part of who you are. Where you're from, where you've lived, and where you've travelled to, for instance, all help tell a story about you as a person.
The way that's represented on Facebook today is the Map on your Timeline. These suggestions help you put more photos on your map so you can share experiences with friends. That trip you took to Disney World 5 years ago may be far down in timeline, but it's going to be easy for friends to find it on your Map.
Hope that helps explain the "why" a bit.
Sad that we've got to a point where we can make this possible but won't because we can't trust pie in the sky.
Torn between the two options and can't decide!
If it's just album name suggestions, I still don't like it, but I suppose I'll live with it.
This is creepy.
Still, I wouldn't expect a digital camera to put those informations in the EXIF (assuming it does have a WIFI and/or an IP address)
Also, to me Facebook doesn't seem to be using any magical machine learning algorithm capable to recognize the location of a PIC from a face, it never did on my PICs, not even when there were landscapes in it.
The guy could instead have uploaded the photo using a smartphone which added the GPS coordinates.
But what do YOU think instead? How does that work? Is it working also for your uploads?