Safari is sending every search query to Apple | Hacker News Reader

Safari is sending every search query to Apple

gist.github.com

327 pointsMrGando124 comments

This is inaccurate alarmism. It shouldn't be surprising that the search bar makes autocomplete requests to Apple as you type. The address, phone number, and map link that popped up when you typed in the name of a local business, for example, didn't just appear out of thin air.

However, it seemed suspicious to me that Apple would make it impossible, as the author claims, to type in the Safari address bar without sending queries to Apple. So, I fired up Charles proxy to confirm my suspicions.

When using Safari default settings, typing in the address bar resulted in a few requests to Apple and Google autocomplete APIs. Then, I turned off "Include search engine suggestions" and "Include Spotlight Suggestions" in Safari search preferences. (Safari -> Preferences -> Search)

As I initially believed, no requests were sent whatsoever when typing in the address bar after those settings were disabled. Can we put out our pitchforks yet, or am I missing something?

Landon feels that disabling Spotlight Suggestions in OS X System Preferences would have an effect on Safari. I'm as mystified as you.

Why doesn't Apple read my thoughts and disable everything?

Howdy,

There are two "Spotlight Suggestions":

- "Spotlight Suggestions" in Safari

- "Spotlight Suggestions" in Spotlight

Both query the same servers, both use the same name, and both return the same information.

A reasonable person might believe that, having followed Apple's instructions for disabling "Spotlight Suggestions" (the Spotlight kind), they'd disabled "Spotlight Suggestions" (the Safari kind) -- especially if you didn't actually see any suggestions appear in Safari (I didn't!).

Mark Rowe, Safari developer at Apple: "That’s probably a fair complaint." https://twitter.com/bdash/status/524005838743035904

There needs to be a single checkbox: "Do not share my Spotlight data with Apple".

There's already a single checkbox for "Diagnostics & Usage Data", and that should be respected too. The network query posted here is actually a search metrics POST, not a live search query, and it's used as metrics for local and remote search performance.

p.s. It was also just pointed out to me that, having selected a specific search engine, a user would not be remiss in believing that their searches would be sent to the only the search engine they had selected.

Perhaps it would make sense to update your Gist to make it clear that Apple isn't trying to force people to funnel their searches through their servers, and that the real issue here is confusing UI?

It's not even that confusing. This is just alarmism for pageviews. Of all the large tech companies right now, Apple has made the most moves to protect user privacy. Disabled 3rd party cookies by default in Safari, randomized WiFi queries, full device ios encryption, opt-in only crash reporting, restricted MAC and UDID access, zero information shared with ApplePay, etc. And yet we're supposed to get out the pitchforks over an autocomplete feature that's disableable with two checkboxes in the UI? Give me a break.

> randomized WiFi queries

You know that the iOS 8 mac randomization feature doesn't actually work right? From imore.com - http://www.imore.com/closer-look-ios-8s-mac-randomization

>alarmism for pageviews

on github...

Real issue isn't confusing UI although that is part of it - the real issue is that they are misleading the user as if their queries are going only where they want - to the search engine they configured. There is no indication that Apple is receiving a copy of every one of their queries to display Maps results that the user may or may not need.

A privacy conscious implementation would ask the user on startup if they would like to include Maps results in their inline suggestions and if so do they agree to sending their queries to Apple.

It's not at all inaccurate.

Might be alarmism, I don't know. But I think it just demonstrates the ways in which users' expectations about privacy are mismatched with the products they use. For example, your average non-tech-savvy user would never realize that an important privacy setting is in a search setting marked "suggestions".

It's hard to say more without seeing the text of Apple's Privacy Documentation that is referred to at the top of the article.

> For example, your average non-tech-savvy user would never realize that an important privacy setting is in a search setting marked "suggestions".

Is this not clear enough?

http://i.imgur.com/i1Zq3E4.png

Apparently not :-)

Those instructions disable "Spotlight Results" only in Spotlight-as-in-desktop-Spotlight.

It sounds like this is what the author of the article was reading. But there's nothing there about Safari.

Let's imagine that the name would be replaced by a slightly let popular company, say one which name begin by an F or a G even one with an M, pichfork would have been already out.

Would you be able to confirm what occurs with the default settings on (suggestions on for both spotlight and Safari) and the browser is kicked into private mode. I'd presume nothing is sent to Apple but can't test at the moment.

Sure: No requests are made while typing in address bar in private mode with default settings.

Deleted comment

> Wow so in other words feature with privacy violation or no feature at all? I.E. if you need search engine suggestions you have to give your search queries to Apple then? How is it not problematic?

How the hell else would you expect it work? It's like saying you expect to drive your car forever and never need to refuel.

> In Yosemite, all Safari web searches are sent to not only the search engine you've selected (e.g., Google, DuckDuckGo), but also to Apple

It would work perfectly fine if Apple only sent the search query to the search engine I have configured. Just like Chrome and Firefox do it.

And then where would the Maps suggestions come from? The App Store suggestions? Have you used the feature at all?

I don't need Apple Maps to suggest anything to me. Where do I go to disable that? I don't see anything Maps related in any of the Safari or Spotlight options. Forcing it on by default so Apple gets all my queries is bad in itself - but not having an easy to find option to disable it is worse.

Edit: Found it in Include Spotlight Suggestions in Safari options - that seems to stop the Maps results. It's a mess - Safari has an option that relates to Spotlight results which invisibly relates to Apple Maps results but there is no mention of Apple Maps in any of the Spotlight options. _And_ it is on by default - not enabled at first run after user consent.

So don't use Spotlight. The whole point is to search EVERYWHERE IT CAN which requires asking a whole lotta services about what you're asking about.

Remember, the UI is intended for most of many millions of users, not the tiny number of people wound up about each and every nuance of a given UI action. Most users either want Spotlight to tell everything it can, or they just don't use it; few indeed want a detailed list of every service involved and a switch for disabling each one (never mind the symbiosis between many of those services).

> So don't use Spotlight.

Oh phew - there's a solution - I can as well stop using a Mac!

>Most users either want Spotlight to tell everything it can, or they just don't use it; few indeed want a detailed list of every service involved and a switch for disabling each one (never mind the symbiosis between many of those services).

I guess there is solid research around this? Good to know everyone wants everything or nothing!

The author of this post is confused about Safari and Spotlight privacy settings.

Spotlight search suggestions are a major new advertised feature in OS X and iOS.

To disable Spotlight Search Suggestions in Safari, open the Safari preferences and de-select "Search > Include Spotlight Suggestions".

To disable Spotlight Search Suggestions in the system Spotlight search menu, open the System Preferences and de-select "Spotlight -> Spotlight Suggestions".

Yep that looks to be spot on based on tcpdumps, although it would be nice to be made aware of this change in the release notes maybe.

Tim Cook last month:

“Our business is not based on having information about you. You’re not our product ... I think everyone has to ask, how do companies make their money? Follow the money. And if they’re making money mainly by collecting gobs of personal data, I think you have a right to be worried.”

“We take a very different view of this than a lot of other companies have. Our view is, when we design a new service, we try not to collect data,”

“We’re not in that business. I’m offended by lots of it. And so, I think people have a right to privacy.”

http://blogs.wsj.com/digits/2014/09/15/apples-cook-your-data...

And ?

Is there ANY evidence that Apple is collecting data. Because it seems to me that all that is happening is that Safari is contacting an Apple service which searches Wikipedia, news, articles etc. Pretty sure it's easy to disable by switching off Smart Search Field in Safari.

Weekly Outrage Time on HN.

Have any of the commentors used the latest Safari? Safari does not passively collect data. It is doing much more. It has what can be called search actions.

Example

Type in a Nissan dealer's name => Brings up with an Icon a clicky that would take you to the nearest Nissan dealer location on Maps.

Type in wiki X => Brings up with wiki page for X.

Pray tell how would they accomplish without processing the results?

I agree that they should have informed users before. But let us not assume something nefarious is happening.

Who's assuming something nefarious is happening?

I find it odd how quickly Apple defenders jump to "let's not assume they're evil" in these cases, when no speculation on motivation is presented.

I personally don't care about motivation. I'm not even sure that a large company like Apple can meaningfully have motivations the way that we as individual humans think of them. All that matters is what they do.

It's obvious from the title that we're supposed to see this as a privacy violation, otherwise it wouldn't be news (or the headline would be about the functionality).

Yes, and I think it is a privacy violation.

However, that doesn't mean it's nefarious. I don't doubt that Apple, if it can have motivations at all, implemented this stuff with the best of intentions. But as I said, I don't care about that, I just care about what they do.

Why do you assume that I assume that you are assuming something?

Where did you get that?

Why do you label me as an Apple defender?

Tell me how would Apple provide this feature without sending data back?

You said, "But let us not assume something nefarious is happening."

Why even bring that up unless you think somebody might assume nefariousness?

Why do I label you as an Apple defender? Because... you're defending... Apple.... Is this a trick question?

How would Apple provide this feature without sending data? I don't see how that's a relevant question. You imply that it's OK for them to do this as long as it's necessary for the feature. I disagree. The feature doesn't have to be implemented. It doesn't have to be turned on by default if it is implemented. And if implemented and turned on by default, they could do a much better job of telling you about the implications and telling you how to turn it off.

"Let us not do X"

can also be uttered before someone does X.

Are you saying that online community of Apple haters (I am not calling you a hater) is not prone to assumptions?

I'm saying nothing about Apple haters. No commentary made there, neither express nor implied.

Even if you trust them to "don't be evil", i.e., don't "collect" it, they could accidentally retain it, get compromised, waste users' bandwidth, etc. All of which encroaches on privacy and personal freedom, even if it can be said they're not "collecting."

See also: NSA's definition of "collect." [0]

P.S. This is assuming the portrayal is accurate. People in the thread are saying it's not. Default-enabled stuff like this is kind of scummy IMO, though, either way.

[0] http://www.slate.com/articles/news_and_politics/politics/201...

This doesn't contradict that. Spotlight provides web search results, how do people think that works?

According to TFA, "In Yosemite, all Safari web searches are sent to not only the search engine you've selected ... but also to Apple..."

So as you say Spotlight web searches have to contact some sort of search engine, but I don't know of a legitimate reason why Apple should need to know about them as well.

Safari also shows the top spotlight search result iirc.

This is (1) in Safari, and (2) with Spotlight Suggestions disabled. Spotlight and Safari should be able to get search engine results or suggestions from the search engine without needing to go through Apple. We can speculate on what Apple is doing with this data, but it certainly seems to violate the user's "right to privacy," if not more.

I'm not sure it's a contradiction either, more of an omission, he doesn't specifically address search data in that interview. However the comments are aimed squarely at Google so the PR message rings a bit hollow.

I don't find it surprising that they're collecting as much data as they can, it just seems odd to call out competitors who do similar things.

The thing about Apple is the positive bias they always get on HN. In these comments:

> Is there ANY evidence that Apple is collecting data.

> I'm not sure it's a contradiction either, more of an omission, he doesn't specifically address search data in that interview

> Doesn't seem like a data collection initiative. It seems more like a complementary search engine

> Do they do autocompletion suggestions like Chrome? That would seem like a good reason to collect this data.

> I actually personally don't mind sending my search data to Apple

It's just amazing. When Microsoft, Google or anyone else does it, it's EVIL and there's no other way to look at it, but when Apple does it, it's somehow perfectly OK, because, you know, they are the good guys.

I guess this is where the confirmation bias comes in.

As someone who would consider themselves to be a realistic 'Apple apologist' (I like Apple, as a company and their products, and tend to assume the positive for everything they do), I see a large negative bias against Apple on HN.

Probably it depends where you come from. If you always spend time on MacFans kind of forums, then yeah, I guess the HN crowd can sound pretty negative to Apple. But in comparison with other stories out there on HN, I feel Apple always gets away with better feedback no matter what they do.

Post seems to indicate that the wrong "Spotlight Suggestions" was unchecked. One in System Preferences (which the post mentions disabling), and one in Safari itself (that the post does not mention).

I would expect the one in System Preferences to disable suggestions in the system-wide search only, and the one in Safari to disable suggestions in Safari only.

Screenshots of the two options: http://imgur.com/a/TWDxy

Google and Microsoft do this too when you enable real-time searching via the address bar, but both are pretty good at letting you know it happens. Doesn't Safari have some kind of first-run experience where the user gets some notice on this?

Google and Microsoft also run search engines.

The App Store is a search engine as well.

Perhaps Apple internally already consider themselves a search engine. This might only be a temporary misalignment of user experience and data being sent to Apple.

Without the requisite user data or some fantastic and soon to be copied algorithm I would expect their search experience to start off about as well as Apple Maps.

You can bet they are going to build it though and it'll look nice.

That might depend on whether they learnt anything from the Apple Maps debacle. They might approach something like that much more tactfully the next time they try it.

With the right expertise on the team and some decent exposure to real users prior to the wide release, they may be able to deliver something that people enjoy using. The micro-features that delight users are something that Apple has traditionally done well, and are also the type of thing that go a long way in a product like a search engine.

In Chrome it only gets sent to Google if Google is the default search engine. I'm willing to bet that IE is the same.

Assuming that Apple does disclose this adequately -- meaning they'd be legally in the clear -- I'd still want to know the answers to: How do I delete the records of my searches saved on Apple's serves? (Google lets me delete my search history.) How do I randomize my unique ID? How many FedGov or state/local agencies have showed up demanding info on which users searched for "dirty bombs" or "how to join Al Qaeda?" Does Apple always require a warrant? Etc.

Separately, Kevin McArthur (author of the Pro PHP book) is reporting this evening on Twitter that Spotlight is harvesting user locations after updating to Yosemite, apparently before the opt-out option is presented: https://twitter.com/KevinSMcArthur/status/523975643340353537 https://twitter.com/KevinSMcArthur/status/523977472627335168

Anecdotally, it sure felt like I had to click "agree" to a lot more during the Yosemite upgrade than any other OS in recent memory.

>How do I delete the records of my searches saved on Apple's serves? (Google lets me delete my search history.)

Google deletes the search history you see but everything is still stored on the servers.

https://support.google.com/websearch/answer/465?hl=en "When you delete items from your Search History, they are no longer associated with your Google Account."

Whether you think that's sufficient or not, at least I have the option. And the last octet in my IP address is automatically modified after 9 months and deleted after 18 months even if I don't do anything. If you can tell me how I can do the same thing with Apple's search log files, and what Apple's IP address retention policy for its search log files, I'd be much obliged...

The DuckDuckGo guys are not gonna be happy...

Sending your searches to DuckDuckGo is OK because they're committed to protecting your privacy.

Sending your searches to Apple is bad because... they're also committed to protected your privacy but they're Apple so it's a big conspiracy.

You may have missed the point. Sending them to DDG is OK because the user _intended to send them to DDG_. That was the entire purpose of the user's searching experience to begin with. Apple being strung into it is different - the user doesn't need Apple's involvement to complete the search.

DuckDuckGo Search Suggestions and Spotlight Search Suggestions are separate features in Safari. Either can be enabled or disabled by the user in the Safari settings.

I use Little Snitch to monitor and control all my outbound traffic. I've noticed a lot more requests to Apple's servers with Yosemite than with Mavericks.

Might this be for sharing one's browsing history across devices? This seems like old news: http://www.tekrevue.com/safari-sync-browser-history/ Perhaps that type of syncing can be turned off? (I don't have Yosemite installed, so I don't know.)

Simple workaround: don't use safari. Long term solution: don't by Apple products. Or at least don't run their software.

Not quite sure why you got down voted. I bought an iPod nano several years ago, because I quite liked the hardware/software that Apple made. However when I found out that they intentionally broke compatibility with 3rd party music software that isn't iTunes, I made the decision to never buy Apple products again.

I don't feel that I'm particularly missing out by making this decision (although I am aware that many other vendors probably have similar ethical problems associated with them).

Can you link to an explanation of how Apple intentionally broke 3rd-party syncing with iPod Nanos? I don't know what kerfuffle you're referencing.

It's long enough ago that a lot of the technical information has dropped off the web (or in some cases been DMCAed offline by Apple[1]), but the short answer is they started authenticating the iPod Nano music database using a cryptographic MAC with a per-device key derived using a secret algorithm. Without the key, third-party software couldn't add or remove songs. They've got more aggressive about it with each generation too; for the 5th gen iPod Nano the code to generate the key had the same kind of obfuscation as their actual DRM code[2], and there's no open source support for the 6th gen because it's just too hard to reverse-engineer.

[1] http://yro.slashdot.org/story/08/11/20/201246/apple-dmcas-ip...

[2] https://web.archive.org/web/20080901013029/http://www.bluwik...

http://boingboing.net/2007/09/14/new-ipods-reengineer.html

So the 2007 version of the iPod refused to work with some random 3rd party software you had, and you've sworn off all Apple products for life?

That's just childish. Not only can companies change but one product doesn't necessarily reflect everything a company has to offer. You should stop being such an ideologue and learn to use the best tool for the job. Which is sometimes an Apple product.

Of you know: http://imgur.com/tfjoo2k an even simpler workaround.

Who's products would you suggest as an alternative?

For browsing, Mozilla is probably the least likely to want to do this sort of logging. There are probably more niche browsers that focus on privacy that I'm unaware of too.

As for other Apple products, you can run Linux on a MacBook quite happily (Linus does, after all) and there are some nice Android ROMs which focus on freedom, such as Replicant.

Epic Privacy Browser - private browser built on chromium for Mac & PCs -- epicbrowser.com (one such niche privacy browser)

[Citation Needed]

Mozilla's "Safe browsing", for instance.

If you're going to go with something FF based, at least choose something with such things disabled by default. For instance, I run PM.

Firefox Nightly also has a tracking protection in about:config => privacy.trackingprotection.enabled its off by default

seems to also block some adverts when on. (probably because those adverts are also trackers)

Wait. People are upset about how unintuitive Apple's preferences are, but Firefox having config hidden away in the advanced about:config is OK?

Nope. Firefox hide away stuff in about:config for testing purposes generally and expose the setting in the user interface after its been tested enough.

Sounds logical enough to me.

Mozilla safe browsing does NOT phone home with sites you're visiting.

Quote:

> Before blocking the site, Firefox will request a double-check to ensure that the reported site has not been removed from the list since your last update.

Source: https://support.mozilla.org/en-US/kb/how-does-phishing-and-m...

Pretty sure it's still actually sending a hash, not a URL

The entity I'm primarily worried about tracking here is Google.

And, pretty much trivially, if the URL was added to the blacklist, they have the corresponding URL to the hash.

(Not to mention, even if they didn't, their core product is based around crawling webpages. I'd be highly surprised if they didn't have hashes of the URLs they visited.)

And, even beside that, URLs are relatively low-entropy. Especially with the path-splitting that safe-browsing does.

I think your fears are unfounded, but it's easy enough to turn off.

Doesn't seem like a data collection initiative. It seems more like a complementary search engine. Apple spotlight search is trying to find other contextual results which may not come back from typical search engines. For example: Apps, Music etc.

I am sure you can directly search on google or duckduckgo to avoid that.

Whatever they do,as long as they make it clear it is fine.I think that's an important thing users should be made aware of.It's not a big scandal but it's good practice whenever a soft does that to have the user's explicit consent.

For instance IDEs often ask if they can submit anonymous datas to a server from the software.

Disabling "Include Spotlight Suggestions" in Safari Preferences seems to make it stop.

FTA: In Yosemite, all Safari web searches are sent to not only the search engine you've selected (e.g., Google, DuckDuckGo), but also to Apple, even if you've disabled "Spotlight Suggestions"

It's because the author is completely wrong.

Disable the Smart Search features in Safari and it won't send anything to Apple.

Here's a tweet from Edward Marczak back in June about this behavior:

https://twitter.com/marczak/status/481818945318428673

with a recommendation to block api.smoot.apple.com.

That seems to be about Spotlight searches, not web searches using Safari.

(And arguably seems like a worse behavior to me, as my Spotlight search terms are often for private info I'm trying to locate on my disk.)

If you're concerned about your privacy, you should look at System Preferences > Spotlight > About Spotlight Suggestions and Privacy:

About Spotlight Suggestions & Privacy

When you use Spotlight, your search queries, the Spotlight Suggestions you select, and related usage data will be sent to Apple. Search results found on your Mac will not be sent. If you have Location Services on your Mac turned on, when you make a search query to Spotlight the location of your Mac at that time will be sent to Apple. Searches for common words and phrases will be forwarded from Apple to Microsoft's Bing search engine. These searches are not stored by Microsoft. Location, search queries, and usage information sent to Apple will be used by Apple only to make Spotlight Suggestions more relevant and to improve other Apple products and services.

If you do not want your Spotlight search queries and Spotlight Suggestions usage data sent to Apple, you can turn off Spotlight Suggestions. Simply deselect the checkboxes for both Spotlight Suggestions and Bing Web Searches in the Search Results tab in the Spotlight preference pane found within System Preferences on your Mac. If you turn off Spotlight Suggestions and Bing Web Searches, Spotlight will search the contents of only your Mac.

You can turn off Location Services for Spotlight Suggestions in the Privacy pane of System Preferences on your Mac by clicking on “Details” next to System Services and then deselecting “Spotlight Suggestions”. If you turn off Location Services on your Mac, your precise location will not be sent to Apple. To deliver relevant search suggestions, Apple may use the IP address of your Internet connection to approximate your location by matching it to a geographic region.

Information collected by Apple will be treated in accordance with Apple’s Privacy Policy, which can be found at www.apple.com/privacy.

If you're concerned about your privacy, you should look at System Preferences > Spotlight > About Spotlight Suggestions and Privacy

The problem is that even after disabling it there, Spotlight still attempts to phone home. Here's a screenshot with "Spotlight Suggestions" and "Bing Web Searches" disabled in System Preferences with a Little Snitch window showing an attempted connection while searching: http://imgur.com/w09aNpz

Thanks, chockablock - I missed that. Same URL to block for both though, apparently: api.smoot.apple.com.

It's good that Google doesn't log your Google searches with Google Chrome.

I wonder if the same behavior is present for the browser in iOS 8. If they have done this on the desktop, it seems plausible they would also have done the same on their mobile platform.

I upgraded to ios8 and the two most buggy, crashy apps are iMessage, and Safari, which have rendered my daily use of the device nearly useless. I gave the hardest time copy/pasting URLs from mobile safari into iMessage, it often pastes in something from days ago instead, no matter how many times I re-copy things I'm other apps.

I want my device to 'just work' like it did when I bought it, updates shouldn't feel like downgrades

Any speculation on what they might be using this data for?

http://www.apple.com/osx/better-apps/

See "Safari: A more intelligent way to search."

Since you asked for speculation, I'd say the NSA. Well, why the hell not, eh!

Out of the corner of my eye, I read that as: Safari is sending every search query to an Apple //c. Everything started making sense.

A few years ago there was a minor scandal about microsoft's Bing "stealing" google results through the internet explorer search box. It seems strange that apple would have that kind of motivations, given that they don't have a search engine of their own... Here's the accusatory blog post google made: http://googleblog.blogspot.com/2011/02/microsofts-bing-uses-...

Do they do autocompletion suggestions like Chrome? That would seem like a good reason to collect this data.

couldn't one just flood this with false positives? looks like a POST to 'api.smoot.apple.com' in JSON format. I'd suspect random entries would make the overall data collected alot less useful...

I'm going to assume that Apple (well, the engineers there) are at least as smart as you and had previously taken this into consideration.

Deleted comment

The gist summary on TFA says:

> even if you've disabled all use of Spotlight Suggestions and sharing of Usage and Diagnostics data

praseodym is pointing out that there are two checkboxes called "Spotlight Suggestions". One in System Preferences, and one in Safari.

It's unclear which OP unchecked.

Screenshots of the two options: http://imgur.com/a/TWDxy

Had deleted my original comment because OP mentioned "all use of Spotlight Suggestions", so I assumed that OP did uncheck the Safari Preferences box. But apparently OP only unchecked the System Preferences one.

As noticed elsewhere in this HN thread (https://news.ycombinator.com/item?id=8480173), unchecking the Safari Preferences box fixes the issue.

This appears to just be Safari syncing with your iCloud Account across all your iCloud enabled devices. That would be why the data is getting sent to Apple.

Lovely. I haven't got Yosemite, yet, but I wonder what happens when you disable api.smoot.apple.com (say via /etc/hosts)?

I did earlier and there is no noticeable difference in Safari.

So nowadays Apple is posing itself as the champion of privacy, and yet they pull this shit...

Safari private browsing is broken too. Just private browse, quit safari, reopen safari and then check in the preferences on the privacy tab and see the cookies, etc.

Reply on news.ycombinator.com