Google's Answer to Siri Thinks Ahead | Hacker News Reader

Google's Answer to Siri Thinks Ahead

technologyreview.com

106 pointsoliversong74 comments

I have been using Google Now for a few months now and it has been amazing.

Example 1: One night I was going to a Roots concert and I went through the venue's website on my Google Chrome on my laptop. Then, right around concert time, Google Now alerted me to what subway I should take to get there.

Example 2: I searched for the Phillies score once on my tablet, and now my phone tells me about the score of each game. Again, I never asked.

Example 3: After 2 days, Google Now guessed where I lived/worked and what time I left for work most days. I don't use any check-in services or anything like that it really just guessed. Then it started telling me what time I'd have to leave home/work to get there on time based on the current subway schedule. Exactly what I wanted, never asked.

Really cool, but am I the only one who finds this creepy as hell?

I've been operating under the assumption that Google (and others) has been building this sort of data up on their users for a while now; I actually appreciate said data being used (visibly) for my benefit for once.

Not really. They (Apple, Google, FB etc.) always had this data. At least they're putting it to good use.

Yes. Your filter bubble won't let you know that anybody else finds it creepy. "If you like, you can repeat the search with the omitted results included."

I'm not trying to be glib, but: so what? Some people will use it, because it will provide value to them. I may be one of them. Kids born today will, if nothing changes, think this stuff is absolutely normal.

Your reaction is a valid one, and I have my limits too, but I think we're beyond the point of merely acknowledging that it's creepy. Please go further: Where is the line? Why? And what are you willing to do about it?

I can't really see Google proper abusing this, as the product is one-to-one the monitoring: commerce. If you accept that, Google Now is just really targeted advertising—a big, NLP-based, bayesian coupon.

It seems like the bigger issue is when malicious parties usurp this data and use it for their own ends. We can look to science-fiction for predictions. Scroogled comes to mind, as well as a (possibly falsely remembered) recent campaign IRL to root out dissidents via internet searches.

And one more scenario: false flag. Since search records are considered evidence, planting this evidence could be all the sway that a jury needs, or even just for defamation purposes.

I think the fear is not of Google, but of making our lives tied to a record-keeper that could potentially have no accountability.

What happens when you perform a one-off search? For example, lets say a friend asks if you know the score to the Yankees game? You really have no interest in baseball, but you do the search anyhow. Should that really influence your Google Now profile?

It does, but you can always remove the team fairly easily.

What, have you got something to hide? :-)

It is creepy as hell, indeed.

Example 3 is very disconcerting from a privacy standpoint.

Mine asked me if it was okay, and I know exactly where to turn it off. It also saw I visit friends at a regular schedule every week, asked if it could remember it, and gives me traffic time estimates to there around the time I need to go, and earlier if traffic is heavy.

I think they require explicit user consent.

Are you sure Google guessed in Example 3? It usually does the "leave now to get there on time" for every event in your calendar (for which it can find some kind of geographical reference, fitting or not).

Yes, if you have location history enabled, it seems to pretty much assume that if you go the same place during the day often enough, it calls it work, and if you spend enough nights at a place, it calls it home.

Google had picked a location for me and called it "work" which was not the exact street address but close enough. I know I didn't ever key in an address and explicitly tell them it was where I work, or else they'd have had the correct address from the get-go.

> ... if you spend enough nights at a place, it calls it home

Finally! We can replace the old cliche where a girl has basically moved in with a guy once she has a toothbrush at his place, by a new one, where she has moved in once her phone calls his place home.

Imagine the heart-breaking scenes when you have to explain to your phone that this is no longer home, but both of you still love it...

Yes, it mines google latitude data for that. Since I left my office job and started running my own business from home. Latitude has been getting very confused about where I work. Last time I checked it thought I worked at the local shopping centre.

Edit: Just checked, it now thinks I work at my kid's school.

You can explicitly tell it where you work FYI: https://latitude.google.com/locationhistory/b/0/dashboard

Thank you! This needs to be much more obvious.

You can also set it from "My Places" in Google Maps.

Your preferred credit card issuer already knows this - if you have a propensity to eat out during the week. It can look at where you are eating most of the days during the week, aggregated over time, to guess which part of town do you work, vs your home (it knows your home address). It can also look at your spend at gas stations over time to see if you own an SUV vs a Car, or whether its a fuel efficient vehicle or not.

Aggregated data from a number of different sources can give a near-complete picture on a customer.

I think Google did guess his home in example 3. It looks like Google Now tries to build a model of where your home is based on your GPS locations during sleeping hours.

The reason I think this is the case is that it kept on updating my "home" while I took a road trip across the East Coast. So when I moved from NYC to D.C. everyday it would tell me the commute to NYC (Home) because I had spent a few days there and Google thought I was based there.

I'm not sure how it figured it out in my case. I turned on the service and a few days later, I noticed it would start alerting me my commute times in the morning and evening. I was looking up something on google maps and it had marked my work and home locations on the map. Kind of creepy, but I'm starting to like it. The way that it anticipates my needs is very helpful.

I work from home but visit my brother's place (in the next city) fairly often. I wonder if Google Now would think I work there. If so, I wonder if there's a way to tell it what's really going on.

I did that a while [two-city] - seems to keep a separate home/work per city.

Also the home/work thing shows up in Google's information about you on Latitude - so you can approve/edit/delete - see http://www.tech-recipes.com/rx/27740/android-setup-your-home...

It thinks I work at my kids' school :)

Here's one that impressed me: I put on my calendar an appointment for a movie I was going to see w/ the name of the movie. Google Now on that day pulled up a card of the the showtimes and theater info for the theater of the movie that I was seeing. I hadn't put the name of the theater in my appoointment.

When I saw the keynote introducing Google Now, the first thing I thought of was that concept video they used to introduce Glass [1].

For example, in the video, the user looked at a subway station entrance, and the glasses showed a rider alert that the station was closed due to a train problem.

I have to imagine that Google Now will be an important information engine behind the "HUD" displays in Glass. Glass is aiming to also follow that principle of offering you information when it's contextually-relevant, and trying to get out of the way otherwise.

[1] http://www.youtube.com/watch?v=9c6W4CCU9M4

So far Google Now is pretty impressive. I love that it will post a reminder in my notifications before an event telling me when I have to leave to make it there on time based on current traffic conditions.

This just happened to me yesterday. I was surprised and delighted. ...Then a little creeped out.

Overall I think it's great though.

This can have some amusing results when the input isn't as Google expects. For example, in my office the conference rooms are named after streets, some of which are also cities. So when I have a 2pm meeting in Folsom, I get a notification a couple of hours before telling me I should leave soon so I can make it to Folsom, CA in time.

This also happens to me. The conference rooms on the floor I work on are named after landmarks in Manhattan. Meeting invites often say something like "2PM in Battery Park". So at about 1:30, my phone vibrates and says it's going to take 25 minutes to take the subway there. Heh.

My horse is named Friday.

Who's on first?

That's cute. Is there a way to tell Google that you mean a different "Folsom"?

Google's Knowledge Graph does have a concept of name ambiguity (as does Wikipedia; it has disambiguation pages). For example, if I do a Google search for "Taj Mahal", the right sidebar shows details about the landmark in India, but below that it also offers to give me more information about the Trump Taj Mahal Casino Resort, or Taj Mahal the musician.

If Google new there was a conference room in a persons current building named the location of an event I would bet it would be smart enough to use that instead. Maybe as more places start using Google indoor mapping this particular problem will be fixed. Most of the errors will likely just be a lack of context provided to Google or a lack of information available in a format Google can access.

If you lived in Great Britain and typed in "Taj Mahal", it would find 100 restaurants within one mile.

While this is incredibly smart and a very impressive application of machine learning, does anyone actually want this? I'm quite capable of looking at traffic reports or bus times myself when I need to. I actually think I'd find this pretty annoying, if my phone told me sports score before I'd had a chance to watch the game, or if it started second guessing me at every turn. This has the feel of valet parking, while it sounds fine and convenient and luxurious, in reality I find it very undesirable and will avoid places that have it.

There's something very much at the core of life in experiencing it for ourselves without having a computer to hand hold us the whole time and remind us it's time leave for work. I don't find it creepy, just embarrassing to feel like I need to be babied so much by technology.

Tbh if I have a calendar entry with a location attached, I'd rather just skip the step of typing the address myself of the computer is smart enough to do it automatically.

I also really like the examples like "how tall is the Eiffel Tower?" Sometimes I want to know a specific piece of information, sometimes I want to read up on something. This allows me to ask specific questions and get poignant answers.

Earlier browsers had one address box, and one search box.

Chrome moved both together - and many people liked it.

I'm sure you like Google Instant and Google Suggest - with the results/search terms showing up even as you type - allowing you to refine your query on the fly even before you complete it!

Google Now is one big leap in the same direction.

I search for a lot of things, and I would like instant access to a lot of things that I've searched for before.

If my phone can figure out what I need access to 'Now' because I've searched for it before or it knows my routing - I say bring it on - make my life easier - and give me more time to do my stuff instead of handling logistics and trivia.

Look at the examples - giving you transit timings when you enter a bus-stop, or giving you traffic conditions to your office or next appointment, or driving directions to the place you searched for. Why not using the super computer in your pocket for that instead of focusing your mind on it and distracting you from your life?

And either you're watching the game NOW, and therefore the score is not relevant to you, or you're not able to watch the game, so the scores are useful. Presumably, the number of people who time-shift a live game is not that high. Those folks can simply delete the card the first time and never see it again.

Google Now is adding a rich information layer to your life via your phone. You can ignore it or you can use it.

I actually hate Google Instant. I like to start typing before I'm done looking at what came up and I like to take notes in the search field. I wish I could turn it off, but that setting never seems to persist across sessions.

Interesting. I'm thinking that Android vs Apple may be decided on the simple fact that sufficiently-better AI trumps any-amount-better UI. Lawsuits and supposed openness be damned.

That actually seems like a really interesting observation. I've never specifically thought of systems in terms of UI v. AI before, but it makes sense if you think of all operations in any computational system as the result of data manipulation by some "intelligence", and user interface as more specifically the interface between that data and user intelligence.

In a computational system, computational intelligence will be used in all instances possible, until such a point as it is either insufficiently advanced or insufficiently informed to continue without making a call to the user intelligence. Thus, the more advanced a computational intelligence and the more information at its disposal, the less interface with user intelligence necessary. By that reasoning, it follows that the ideal or perfect interface would be one which treats the user purely as a data source (more precisely, as the absolute last data source in the pecking order – one to use when absolutely no other data source accessible could possibly provide the same data or make the same decision with reasonable confidence).

In that sense, it's likely that going forward we'll see UIs moving more in the direction of Now / Wolfram Alpha / Siri and using such mechanisms to manipulate and return greater amounts of information.

Butlers had a pretty good UI.

A couple more thoughts:

* This idea of data being acted upon by intelligences is essentially a generalisation of the canonical algorithms and data structures.

* The argument presented would also explain why anyone accustomed to a certain command line interface will swear by it over most graphical interfaces to the same system (the reason being that commands are usually a much more direct interface between the system and the user intelligence).

One more detail to throw in: the user as data source will move toward expressing what they want rather than how to get it.

Which is why it seems to me that introducing Siri was a dumb move for Apple: UI is one of Apple's core competencies, but machine intelligence is pretty much the one thing Google is best at.

But it was smart of Apple to get their product to market first. Waiting for it to be "finished", then being stuck behind Google in ML technology indefinitely, would have been dumb.

You mean like relying on google maps initially and getting massively stuck behind them when their own is released?

I think the UI gap between Apple and Google is either closing rapidly or has already closed. Android looks slicker and more useful every day while iOS is slowly grinding on with the same old app icons and home button.

I think Android 4.1 is finally something like the "agents" people have been envisioning for over a decade.

The possibilities on this are endless and currently Google is about the only company that is able to do it (from my point of view). Google knows my calendar, Google knows my contacts, Google knows pretty much everything I buy online (because they handle all my emails), Google knows about my travel plans (email again), Google know what I've been looking, where I've been (Latitude) and so on.

There could be also huge opportunities for money making in all this. One good question is should they keep all this just to themselves or would it be better to try to create an ecosystem for applications.

Let's the take travel plans as an example. It would be fairly easy for Google to mine my calendar and emails to figure out my travel plans. This would open up opportunities in offering me flight upgrades and other travel related services. Google could either try to use this information themselves, they could allow advertisers to use it, or they could expose it to 3rd party apps (of course with my consent). Keeping with the agent idea these things would be much more than just highly targeted ads. They could be direct offers, like "Say Yes Please, to upgrade your flight from Helsinki to Barcelona to business class for 150€".

So I can steal someone's phone, and just have it remind me of all that user's private info by walking around? Nifty.

Edit: I know the above is glib, and I'm not trying to rain on the parade of unbelievable achievements in data-wrangling that are going on here (I'd love to hear how it's done computationally), but I don't like knowing that all the privacy controls something like this deserves will be rolled out hastily in response to someone's life inevitably being ruined by the careless trust of one's life to a device that can fall out of their pocket. It's easy for the savvier of us on this site to say "serves them for not thinking to take precautions", but when we make it so easy to give a third party the reins, even we get complacent.

That isn't a problem with Google Now, that's a problem with smartphones generally. Even without Google Now, you can log onto someone's Facebook account, read their emails, read their texts, view their calendar, etc.

Assuming you could unlock it, yes. But this has been true since the age of Palm and Blackberry phones.

With a little imagination, it's not hard to see why some are nervous about a search engine that stores every interaction in an attempt to profile its users and guess their intentions. Don your finest tin foil hat as we channel surf some internet TV stations from the future:

LIGHTS DIM; THE GLOW OF A COMPUTER SCREEN FILLS THE ROOM

"In what is thought to be a world-first, a man's life was saved late Friday following automated intervention by a search engine.

"Close friends of Mr Exampleson, who BBC News can now reveal were the only members of his "Best Friends" circle on Boogle Plus, said they were able to get to the Beachy Head cliffs in Southern England five minutes before his arrival thanks only to an email from a 'Concerned Partner'.

"The email said that their friend was displaying worrying behaviour, that he had personally disclosed his intention to take his own life, that he had planned a route to Beachy Head with an estimated arrival time of 19:27, and that he was currently on the A267 heading South at a speed of 57MPH.

"A spokesperson from Boogle Ireland this morning said that they have long been publishing helpline numbers on their website when a user enters certain search terms, and that this was simply an extension of that service.

"Privacy advocate Mr Foilhatison joins us now to discuss what this means for..."

SWITCH CHANNEL

"...surrounded the building and ordered the suspect to come out with his hands up. When questioned, it emerged that the man was famous writer Real Steveyson, who is staying in the area to write his hotly anticipated thriller, Cryptonomnomnom. Mr Steveyson has since been released with a public apology. Police are now said to be following a variety of other leads they hope will 'prove to be less of a hopeless time suck'.

"Asked why he felt detectives had singled him out and surrounded him in his remote location, Mr Steveyson said that he had been searching Boogle over a period of two months for various methods to reduce DNA evidence at crime scenes for his upcoming book. He had also been Boogling for local peat bogs, completely unaware of the recent bodies that police have pulled from them. Mr Steveyson suspects that Boogle may now be sharing live search information with local law enforcement agencies in a bid to both pre-empt crime and increase conviction rates. The idea sounds so far-fetched that he plans to make it the subject of his next novel, which he hopes to publish for the first time via telepathic transfer using the new...

SWITCH CHANNEL

"We're here in the diamond district where Miss Caratson is closing her shop for the last time. Tell us, Miss Caratson, is it the same story for you as it has been for the others?"

"Sure is! I haven't sold a single engagement ring since Boogle Hive got popular."

"Can you explain why?"

"Boogle talks my customers out of the purchase! Somehow it detects that they're in a jewellers. And that's when it gets really weird. It always starts the same way: they get that dreaded audio alert. That's when I know I've lost the sale.

"One time, it told a young man that, based on a language analysis of all correspondence with his partner, there was a 76% chance that his fiancé-to-be was only interested in him for his large Boogle Wallet balance, so he might want to hold off on his big plans.

"Another time, Boogle tells my customer that, based on something called their "Boogle Lifestyle Score", they probably weren't the marrying kind. Then it reels off a list of 12 ways they'd have much more fun with the same amount of money, and it lets them know that six of the 12 things are within a five-minute walk.

"And don't get me started on the glasses! Guys using those Boogle Glasses always get told that Boogle has found the same ring they're looking at 500 bucks cheaper three shops down. It's a race to the bottom. I just can't win.

"Some of us tried to circumvent it by installing Faraday shielding and blocking the damn system altogether, but Boogle just warns people to avoid us before they even get in the door. It's over. I'm done!"

"What will you do now, Miss Caratson?"

"Isn't it obvious? I'm going to do what everyone else is doing. I'm going to start a..."

SCREEN DIMS, AMBIENT LIGHTS FLICKER ON

Yes, some of this may be a little far-fetched, but the idea of a computer network attempting to infer intentions based on a lifetime of intimate accumulated knowledge is troubling for many. Even if such a system has the potential to be incredibly useful, I suspect that the amount of blind trust it demands would make it a challenge to adopt for more than a few of us.

Wow, that's a pretty good one. Why don't you expand it a little and make it into a short story :)

Recently, my parents went on a trip to London. Before they returned, I received a Google Now card with their flight information and estimated time of arrival. I am not sure if they gathered this info from email or sms. It was somewhat creepy, but certainly helpful and convenient (I needed to call them before they left).

I have been quite pleased with Google Now and use it much more than I ever used any of the other Siri clones. When I am busy, I forget things easily and it is nice when Google just gives me helpful information to keep me on track.

Interesting, would appear the AI is based upon your email and general google collected info upon you. With that in mind I better make sure I'm catching all the spam comming in as one walk near Soho in London and things could get interesting.

From what I have played with google now and its card alerts it has so far impressed me with bus options to get home, though mostly when I'm upon a bus already and it offers alternatives. If the bus breaks down then I'm covered is how I see it in a thru positive blinkers.

The trick will be for them to be helpful without being creepy.

There is a certain threshold where helpfulness and creepiness cross, specifically when it will be so helpful that creepiness will need to be ignored. At that point it becomes dangerous to rely on it (see: drones and their exploding popularity in civilian applications).

Google Now is awesome.

Search for a place on Google - card shows up with driving directions on phone.

Morning and evening, shows up driving time/directions for work/home.

Why are you people thinking this is better than Microsoft Words annoying Clippy? "It seems like you are writing a letter..."

I already know how to get to work every day.

The difference is Clippy tries to grab your attention, most likely without making a contribution. Google Now doesn't try to grab your attention, just show you information that might be helpful.

It only shows up if there are traffic delays or public transit schedules.

It's out of the way - at best there's an extra notification, or it's one card in Google Now - when/if you go there.

Extra information when presented unobtrusively is not annoying.

a. Do you know how long it's going to take today? If there're any specific traffic issues that are causing delays on your usual route?

b. What if you have an appointment today to a different place than work - wouldn't you like to know the route there?

c. If you take transit, wouldn't you like to see bus routes when you reach the bus stop, instead of manually searching for it?

Think of it as your Personal Assistant... Not sure why you wouldn't want one that's always with you and can be flicked away if it's not what you want [permanently or only this time].

So the voice search is called Google Now? During the keynote it was confusing because they kept calling the voice stuff "Voice Search" and "Google Now" was only used when talking about the cards (which are confusingly in the same app).

The voice search is not called Now (it is just 'Google Voice Search') but since Google Now and Google Voice Search were released at the same time they are commonly confused.

You've got it right, Voice Search and Google Now are separate entities living in the same Google search app. IIRC the keynote introduced it as something like "The New Google search = New UI + Voice Search + Google Now", so no contradiction there.

People just happen to like calling the entire app "Google Now" since thats the part they remember, so it became tradition. Much like how some people call the iPod Touch the "iTouch".

Voice Search is Voice Search. Google Now is just the predictive notifications and the cards (the notifications being the more signifiant part). The two are separate.

Voice Search has been available since Froyo, IIRC -- although in the past it would simply show search results rather than respond with a voice for anything other than a small set of phrases. Voice responses for more general Knowledge Graph queries is new in Jellybean. (Long-press the search button on an older phone, like a Nexus One, to trigger it.)

Not looking forward to the ads. I'd rather pay for it.

I would pay for any service if I had guarantees that my data would not be sold or used for third party profit in any way.

Reply on news.ycombinator.com