Not OK, Google | Hacker News Reader

Not OK, Google

techcrunch.com

Google Now has done the same for me, told me how long it will take to get to a bar I frequent. My reaction was quite exactly "Oh that's neat, thanks!" and I went and had a great burger that night.

Totally OK for me Google. I respect that people have different privacy thresholds, but I think the fact that it's different for everyone is being lost in articles like this.

One tiny caveat being that, Google (and others as well, to be fair) will be able to indirectly collect data even on the privacy-aware part of the population who don't use Google much. The simplest example being, even if you don't use GMail, still some part of your emails inevitably end up in GMail inboxes. Now also consider this: just being a guest in a Google-stuffed house means you are under surveillance.

So no, it is not just my problem or your problem, it's everyone's.

Sure, and to illustrate your point, I have an email very similar to someone else's. I very frequently get their emails (invoices, church events, travel itineraries, purchase receipts). Google thinks they're _my_ trips, and updates me about flight times.

I think this example serves both our points. To your point, it's totally leaked this other person's info into my "google world" because I'm on gmail. On the other hand, that person is leaking his information directly to me just because of typos when he fills out online forms. Perfect privacy requires a lot of vigilance in a digital world, with or without google/gmail/hotmail/yahoo/etc.

If you visit a place of business you are potentially under surveillance. I'm sure there is a distinction but I'm failing to come up with it right now.

I was somewhat radical about privacy in the late 90s (only person I knew that read every EULA) and am still a supporter of the EFF but I don't really understand the issue here.

Google has top notch practices in my opinion regarding privacy. If anything I would be more worried about smaller companies with shadier practices and lower security standards holding my personal information. There have been seemingly illegal practices from companies like Sears with how personal data is collected and used. It's easy to throw shade at a big companies, write a sarcastic title, then get clicks.

Example: "Intuit’s TurboTax stores highly detailed financial data for millions of users who import their W2s, their banking data, info about their mortgages and more. Right now, all of this data is locked into TurboTax, but the company is now thinking about how it can do more with it by giving its users the option to share this data with reputable third parties." ... https://techcrunch.com/2016/09/22/intuit-wants-to-turn-turbo...

Perhaps we need to start normalizing encrypted email. Just like https everywhere is no longer considered necessarily "tin-foil hat" SOP, moving this direction for email needs to be socially normalized.

Going further, given that an encrypted email to Gmail will simply be unencrypted and then available to GMail, include in the protocol authorized (via both white and black list means) agents of the recipient. So, if you are hosting your own email but the intended recipient is expected to be not hosting their own email, the sender can blacklist "agents" such as Gmail and Yahoo! Mail, or blacklist all except for those chosen to be white-listed such as Proton Mail.

I'm not sure you are any less responsible for your own privacy despite the fact that companies like Google as well as others are making it more challenging. The example you mentioned seems easily fixed by using GPG.

Granted there may be a place for regulations to help us restrict what companies are able to do(perhaps making it easier for you to identify a region that is being recorded, right?), but, at some point society can't help the fact that you'd prefer if machines were unaware of your existence. That's just something you have to solve for yourself.

If you don't care for the products or services, you could always opt not to buy/use them. That includes avoiding areas where Google appliances are present. I'm fine with trading privacy for AI-driven convenience, though I already have a phone I like and an Echo, so I have no compelling reason to buy these particular products right now.

a non-insignificant proportion of hacker news seems to actually work at google and run to their defense at anything somewhat critical of their overlord.

I fear you will be unable to recognize when that burger was your choice and when it was a reaction. You probably won't notice. And that is harmless.

I also fear you will be unable to notice in which areas of life and information the distinction between choice and reaction is harmless and which it isn't.

Of course, I'm not talking about "You" you, but just people. Me as well. I feel we are widening the field of unconscious decisions and I see that as inherently bad - in my fellow humans as well.

You could say that Plato wanted us to make easy things simple (link for distinction: https://www.infoq.com/presentations/Simple-Made-Easy).

I believe this to be a move in the opposite direction. We should have a care.

I'm sorry but that just sounds like blind fear mongering. What you're saying is vague and doesn't really mean much.

It's like saying we shouldn't use prescription glasses, or medication, or cars, because it's not "us".

Humans invent all these tools and systems to improve and optimize our life. Make our vision better. Make our health better. Make us move around faster. In the case of AI, make us do perform certain things more efficiently.

Imagine it wasn't an actually computer. Imagine it was a personal secretary you had that gave you the EXACT same information. Gave you your flight information, turned on the light when you asked, gave you the weather and your schedule. Would you think that was wrong? That this isn't "you"? No, it's just optimizing your life, but now available for a wider population rather than rich people.

"Imagine it was a personal secretary you had that gave you the EXACT same information. Gave you your flight information, turned on the light when you asked, gave you the weather and your schedule."

In your metaphor, you are implicitly paying the secretary, so the secretary is incentivized to maintain your interests.

How much have you paid Google for its free services?

Your metaphor is inapplicable. You don't have a secretary telling you these things; you have a salesman trying to sell you things, and the salesman is getting smarter every day while you aren't. Not the same thing at all.

What he's saying is that this is not "humanity inventing something to make life better". It's a company inventing something to make money.

And it's not a simple product like glasses where you pay with money and then they improve your vision. It's a product which goes far beyond your understanding and for which you don't pay money.

Google isn't interested in making your life better. What they are interested in is getting you to believe that they want to make your life better and to then recommend going to that bar, because the bar owner has given Google money to advertise for the bar.

Yes, you might actually like that bar, but Google isn't going to recommend going there in intervals which are beneficial to you. They'd rather have you go there a few too many times. Because that's what makes them money. It's not improving your life, which makes them money. Their AI will always work against you, whenever it can without you noticing.

Imagine that you were trying to quit smoking and your electronic secretary kept updating you on the cheapest place to find your favorite cigarettes? With no way to tell it not to do that.

I think you misunderstand me by a large margin.

I'm not saying we shouldn't use AIs. We should, however, think about how we use them.

To build on your example, what are the dangers of having a personal secretary on the payroll of anyone but you?

What I am expecting from this is a super devious filter bubble - because that's how you make money. Google's old slogan "Don't be evil" is long gone. "For a greater good" might be more on point.

>In the case of AI, make us do perform certain things more efficiently.

What does the Google Assistant help me do more efficiently? In all honesty, I can't figure it out. I don't need or want a secretary, and I can do written planning for myself.

I need less paperwork and fewer web forms and identities, but the Google Assistant only promises more of that crap.

I'm never buying one. It's a sacrifice of privacy for zero to marginal gains in convenience.

To my mind, leading a simple life is enjoying a burger at a restaurant/bar I frequent already. Simplicity _is_ accepting that Google algorithmically noticed a trend and just helped me do things I already do.

Yes, traps are usually designed so that it is simple to get into them. It is not that cheese is bad, it is that you are trapped.

Before replying, you could at least have made an effort to understand what I meant with the distinction between simple and easy.

If you do not care what I say, why even reply?

Sorry, who's mind? It sound like you are renting it out.

That's the thing though. I reject the notion that you ever actually make a choice. I would posit that 100% of the actions you take are simply the deterministic reactions when the current world state is filtered through your brain. Then, after the fact, your brain gets busy inventing a reason that you took a particular action and calls it a "choice" when really you were just going to do what you were going to do anyway.

"I ordered this burger because I was hungry and it tastes good" vs "I ordered this burger because Google was able to successfully predict that I would be receptive to having burgers, or the idea of burgers, placed in my environment"

Sure, but philosophical musings on the nature of free will aside, there's a practical worry about the amount of power a private company has over your actions. I'd rather be ordering burgers because they taste good than because a company wanted me too--I expect this will lead to greater happiness for me in the long run.

In effect you argument is that we don't have free will, right?

I wonder what is then causing inefficiency when we read restaurant's menu and can't decide what we will have.

I'm with those who think we make choices and decisions far less often than we think, but that we still do make them.

I think for quite a large number of people, allowing AIs to make decisions for them will probably be better for them.

When the AIs are working in service of corporations this seems incredibly unlikely.

We already see what happens when peoples decision making is coloured by mass media advertising. An obese population trapped by debts taken out to fuel consumption.

It is in other peoples best interests for you to work like a slave, be addicted to unhealthy habits & run up vast debts in order to buy their products.

We keep allowing those with power to distort the markets gaining themselves more money and more power at the expense of the little guy. I don't see any reason why AI in the service of the powerful will do anything but accelerate that.

Imagine some day your doctor advises to cut back on burgers and alcohol. Is Google going to incorporate that advice in its bar recommendations?

That rather depends on the objectives of the AI.

If you replace "AI" with "marketing" would you still make that statement?

don't you think thats a pretty severe statement wrt to free will and agency? if i'm just a consumer wired up to a machine thats deciding whats best for me (even with the best of intentions), doesn't that make me less human?

should I just be a actor playing through a set itinerary of vacations and movies and burgers and relationships? maybe you think its that way already, except less perfect than it might be, but thats a pretty frightening notion to me.

The same argument was historically made to justify slavery.

Given all the other points in life where, despite my awareness, I don't have much choice, how is an AI just directing me really any different?

My culture, education and skills limit what work I can do.

Our culture places limits on a vast number of experiences. On the road and the only thing is fast food? Welp, eating fast food. Live somewhere that only has one grocery store or cable provider?

I don't really see AI in the form Google is peddling as really all that much different. We're just 'more aware' that the world around us is really guiding us.

I may be somewhere new, and can only see the immediate surroundings without a lot of exploring. And let's be real, in the US, most cities are the same when it comes to restaurants/hotels and such. There are differences in culture but we don't usually see them if we're just visiting. Not in a way that matters.

Google will let me know that the things I prefer back home? there are equivalents nearby.

Fencing ourselves in is what we do. Who knows, perhaps a digital assistant would help us stick to our personal goals and decisions better. Rather than just having to accept what's there.

Almost all decisions are unconscious decisions. whether or not Google is involved. We usually rationalize our reasoning after the decision is made.

> I feel we are widening the field of unconscious decisions and I see that as inherently bad

I'm curious why you think this is bad. I don't necessarily think it is good but I also don't necessarily think it is actually happening

Which news-sources do you use?

Which news-sources are you going to learn about?

Which news-sources are you for some reason very unlikely to encounter?

Now apply a real-time AI filter-bubble, able to also include government policies in its decision-making, onto those questions.

I believe the most important thing in life is thinking. I believe a key element of thinking is looking at "easy stuff", the stuff we just live with every day and don't think about, and for some reason be forced to think about it and make it simple.

Take the Snowden-leak. We lived a nice life being the good guys and that kind of surveillance was publicly thought of as conspiracy theories. Suddenly we were forced to look at what was going on. How much of it are we okay with? On the grounds of what principles and tradeoffs? This is all very unpleasant, but we're all better off for facing those questions and work towards new principles. We take a chaotic gruel of cons and pros, and try to hammer them into a few simple principles our societies may function by. For instance, the separation of power into 3 has served us well.

I fear that we end up in a world where raising such unpleasant questions becomes almost impossible - and we'll never even notice. Not because of AI (I believe AI to be inevitable and fascinating) but because of the way AI is used.

Living a life assisted by an AI, made and paid for by someone else, seems like the epitome of naivete to me.

> I fear you will be unable to recognize when that burger was your choice and when it was a reaction.

Maybe the illusion is that it was a choice . . .

Not far from the mark. People have quite different behaviors when asked "what do you want?" vs a constant stream of "do you want X?" questions.

What I want mainly from Google is more and easier ways to customize my level of privacy. The article touches on the EFF's stance against incognito modes briefly, but it's an important one; I don't want lack of monitoring to be something I start a separate session for, with a logo of a creepy dude implying I should use this only for spying and pornography. I'd like to get as close as possible to an assistant that remembers relevant data on where I go and how long it takes, but ignores my browsing history to psychologically manipulate me into buying things--of course, that needs a different revenue model.

Does https://myaccount.google.com/activitycontrols?pli=1 do what you're asking for? It's fairly fine-grained.

My experience is that Google app and system updates have a tendency to force/trick/nag you into giving up privacy.

As example, the old weather widget from Google's "News & Weather" was replaced by Google Now. That provided a similar experience for some time but then stopped working with another update that required search history to be enabled and/or some other setting in privacy control.

Also the launcher integrated Google with a system update(Moto G line of phones). I have since replaced launcher, browser, search app (all with open source replacements), weather app (with a paid service). Convenience has suffered...

I love that this uses 'pause' vs 'stop', 'off', 'disable' 'deactivate'.

Your location history tracking is paused..

As was mentioned below, in Google's case I'm particularly looking for ease of use and a UI that lets me turn things on and off quickly and intuitively. That might sound odd, as if I care deeply enough about my privacy to complain about it, but not enough to quickly switch the settings on my Google account, but I do think it matters.

The more specific work I need to go through to set up my privacy, the less inclined I am to do it. If I didn't think I was able to be manipulated psychologically in this way, well, I wouldn't worry about advertising at all! If I were to ever do something politically dissident/personally embarrassing on the internet (not that I ever have of course) I'd go to the trouble of ensuring encryption and being hard to track, but I think it's important that I'm able to say to Google "Hey, I'm cool with you telling me when I should check off work to hit the bar, but it's super weird that you know what I should get my Mom for Mother's Day."

Of course, the simplest way of making a system that's both fine-grained and intuitive might involve... more AI, so I'm not sure how to crack that issue.

You shouldn't have to spend that much time managing your privacy.

I think eventually they add so many capabilities and so many fine-grained controls and it becomes impossible to manage the UI or to find the right options. Even looking at the settings for Android privacy / settings, it's pretty hard to find anything.

This is by design, so that the majority of users are confused and leave the defaults as is, enabling Google to do whatever they like.

> logo of a creepy dude implying I should use this only for spying

I never thought of that before. But, what a subtle way for Google to dissuade people from using a tool that could impact their revenue.

When it first told me it knew hiw long my commute would take, I realised it was creepy as all hell that the people (in another country with few protections on data) providing my phone software knew enough about me to tell where I worked and when I was going there.

And it annoys me that on maps, when you turn off all the spying capabilities there's no fallback to local history. You either share it with us or you get none.

  > ... it was creepy as all hell that the people [...] knew enough about me

No people know. Software metaphorically ‘knows’, but don't anthropomorphize it (it hates that).

Failing to provide local history is essentially one of the dark patterns to getting you to turn on their privacy collection. Most things Google requires it for could easily be done outside the cloud, but by making things depend on the cloud, and then telling everyone you can only do it with the cloud, you convince people that they need the cloud. When in reality, they never did.

GPS navigation devices with much less storage than a phone have been more than capable of what Google Maps offers for a long time. There's essentially no reason for it to do anything with the Internet except getting map updates.

It may be OK for you but there are at least three real concerns here:

1- There is no way to set your privacy level.

2- Things that Google/Siri/Alexa know about you are not limited with the name of the bar you go frequently. They know much more about you. And you don't know what they know. The sky is the limit here.

3- Things that they know are not limited with you personally. They know about you, your family, your friends and all their interactions. They know very much about the whole society.

1 - Sure there is, Google has fairly fine-grained tracking control. Not perfect, but as another commenter noted, this is a double-edged sword, as _too many_ controls can conversely hinder user control (see Facebook's privacy revamp)

2 - My point is that I personally am OK with Google's AI knowing more about me. I respect that others aren't. I'm not naive in my acceptance.

3 - I don't really have a response here.

Sure there is, Google has fairly fine-grained tracking control.

The privacy control where I disable location tracking and half a year later when I look in Google Dashboard I see months of travel history?

I respect that others aren't. I'm not naive in my acceptance.

So what do you do in a situation where your use of Google's data collection also affects people who do mind it? I would not be comfortable visiting a friend with an always-listening device like Alexa or Google's equivalent.

I nuked my paid Google Apps account a couple of months ago. I had enough of their total disrespect for privacy. E.g. conversations that I had in Google Mail (which is protected by the Google Apps agreement) were used for suggestions, etc. in Google+ (which is not covered by the Google Apps agreement and uses data for targeted advertising).

I don't know about Google, but I know that Siri and Alexa only collect and send data when you ask it to.

You can monitor the traffic of the Alexa and see that it is only sending data when you ask it to do something, and furthermore, Amazon gives you a log of everything you've said to it and it recorded.

And you can also get a log of everything you've ever ordered from Amazon, but there are still loads of other signals which aren't visible to you, as evidenced by the fact that you browse a product on amazon.com today, and now for the next week, you see that product advertised back to you all over the web.

Others may be reviews left, reviews voted on, prime video consumed, audio/video/book samples consumed, kindle activity, how long you spend on a product page, how you scroll on it, the breadcrumb of how you got there, and surely dozens more.

Home claims this as well, and I for example have a log of audio clips and transcriptions of all of my google now searches.

"Hi Ubercore, Google and your health insurance company here. We are worried that you are frequenting a facility that serves too much alcohol and wings. We care."

I really wish this was up top. I can't believe that the top rated comment on HN is a thinly veiled "if you don't have anything to hide, you don't have anything to worry about".

My impression is that (in the main) younger people have lower privacy thresholds than older people. Not for everyone (of course). Just on average.

My impression also is that most early adopters of this kind of technology are younger people. (again: mostly)

So this brings up an interesting question about the future. As the young early adopters age, what will happen?

a) their privacy thresholds will also increase and they will have a "oh holy crap" moment in the future, where as a middle-aged or older person, who has lived a now much richer and problem-laden life, they will realize that google (and/or other co's) have what they consider now, as too much personal information about them,

b) they will keep their young-ish privacy thresholds as older people, and in general, across society, people will have lower thresholds than exist nowadays. In other words the world will change.

My money is on a)

Conversely, there are a fair amount of older folk who are very much okay with government surveillance and a general lack of privacy (particularly our inherent rights) because, e.g., they have "nothing to hide"

My impression (in the main) is that younger and older people have different views on privacy. Older folk might be creeped out by Google knowing their schedule, but okay with the NSA or FBI or whomever reading their emails because "because terrorists" whereas younger folk are more likely to balk at the latter, but very much okay the former.

Do you think my opinion is accurate? I'm curious because to some extent I completely agree with you.

I don't think A will necessarily happen for most. We are a product of our experiences. If, as we grow up, we get comfortable using always-online technologies and never suffer any consequence from those experiences, I don't see what would motivate us to suddenly doubt these technologies. I am confident B is the most likely situation; that's how societies move forward so quickly with tech.

I met a good friend and current employer because Google Now told me about an event I might be interested in a couple of years ago.

Of course it also once told me how long it would take to get to an ex's house from my current girlfriend's place.

How about an 'OK, funny once' command?

As a possibly humourous thought along those lines, imagine explaining to your future child they were born because of a Google Now reminder. ;)

(eugenics through large scale suggestions anyone? ;>)

  > How about an 'OK, funny once' command?

I hope that's a Heinlein reference.

My experience has been quite the opposite in terms of convenience and relevance: I commute by car and train (for some parts of the same journey) and google Now, google Maps (etc) have been totally useless there: telling me about traffice jams when I'm in the train, not telling me about train delays, etc... it now somehow thinks my home is at the train station, it tells me the last bus home is leaving soon when I've been home for hours, also Google Now's insistence on bombarding the leftmost pane on my phone with the most click-bait articles ever, often about things I had a passing interest in months ago, is just laughable. I would be glad to give Google some of my very precious privacy, implementing some countermeasures like multiple and burner identities as needed, if I thought they had any chance of actually providing real value. So far they have failed miserably, I am not sure the economics of providing a really useful service there for free just with marketing information as source of income work now, or for a very long time. You'd need strong AI to actually help my day to day life, with solid non obvious guessing based on many very local and specific factors. I guess as long as people kind of believe that this future is coming, they may tolerate the invasion and forget about the promises.

To me it's pretty simple. If you don't like what Google is doing, don't use their services, and don't buy their products.

But as mentioned above, if you don't want Google to potentially track your behaviour and preferences, "don't use their services" encompasses "don't send email to anyone with a gmail address".

The sad part is that the user turned off Google Now because he didn't want google to know about the bar he visited. Google was tracking and recording his location before Google Now, he just didn't know it. It's still tracking it after he disabled Google Now.

The only difference being that the user now believes that she isn't being tracked now.

I hope that there will be a day when Google and Facebook will combine forces and work on a sequel to "The Lives of Others" [0].

[0] http://www.imdb.com/title/tt0405094/

Yeah, but that is also the equivalent of 24/7 surveillance of all locations you visit. Google will end up figuring out whom you sleep with, etc. from that information.

Pretty much your only privacy is in your head at that point.

I'm not sure that is a "threshold" of privacy but rather a "I am okay with 24/7 surveillance of all of my activity."

hypothetically: you express radical political ideas to your friend with the expectation of your statements remaining in confidence-- but google was listening. now, your feed recommendations steer you further down the path google thinks you were already on. you are ready to attend a protest and perform civil disobedience, as google now knows based off of your interest in what it has been suggesting to you. it suggests (as facebook does now regarding making events for birthday parties etc) that you and some other people form that protest, and, because it said so, you do it. except it's a trap. the police's google feed tells them that some undesirables have planned a protest, and you're imprisoned.

is this story unrealistic, or has it already occurred?

I remember the huge smile I got on my face the first time Google Now picked up on the fact that I went to the same bar every Wednesday evening.

One Wednesday afternoon, at work, I got a notification saying "Travel time to the Lion & Crown". The first thing that ran through my head was "oh my god, I'm living in the future".

I am actually quite uncomfortable that my stock Android is making suggestions on how long it should take for me to get home or work (when I have never explicitly mentioned that it is my home or work).

The problem is that I want to use Google Maps so what choice do I really have?

Sure I use a dedicated gmail for my phone but that really does not help much.

HERE is an excellent maps alternative, I'd highly recommend it.

You should be glad that it doesn't remind you:

"Hey, it's a been a while. Why don't you go to ... today? Traffic conditions are favorable too."

Their business model is to attract and sell your attention to other companies.

Given that, and the existence of pervasive surveillance and data mining, the above is inevitable.

Personally, I wish it did have reminders like that. Offer me a coupon too, while you're at it.

I would not want Google knowing I sometimes drink too much, or that I do so and get behind the wheel of a car. Easy inferences it could make, given the time I spent at the bar, and the purchases on my credit card. That could even have consequences for the cost of my auto insurance. Edit: and health insurance.

But wouldn't that be better for us as a society to know? The outcome sounds positive here.

I actually thought the most interesting point this article raised - for me at least - is the implict branding associated with the "OK Google" command. All privacy concerns aside, if I'm going to have a "personal Google" I want to be able to thoroughly personalize it.

but I think the fact that it's different for everyone is being lost in articles like this.

I think that it is different for everyone is completely and utterly obvious. It's clear the author doesn't think it's OK, but that it's his/her opinion.

This location history is really bad though. I added a test gmail account to my device a few weeks ago, but didn't remember location tracking was a per-account setting - now Google has nice big logs to hand whoever wants them and I can only delete them on a day-by-day basis (from the Android app at least).

Extremely annoying. This sort of thing should not be acceptable, an honest mistake results in every place I've been being logged in such a way that anyone with access to my Google account, access to Google servers or with a subpoena can have my full location history in a matter of seconds.

This needs to be a big red option every time you add an account "we're gonna log everywhere you go and hand it over to whoever we feel like, you cool with that?". It'd be different if the log and analysis were done only on my device, but doing this on Google's servers is completely unacceptable by anyone with even the weakest standards of privacy.

Find Google settings in one of these places (depending on your device):

In your Settings app Settings, scroll down and tap Google. Open a separate app called Google Settings Google Settings.

Tap Location and then Location History.

At the bottom of the screen, tap Delete Location History.

Also https://maps.google.com/locationhistory allows for delete all history.

just having a mobile device you're being tracked in Australia now the Government now also has access to this data from the network providers the only way to be free is to not carry these devices

What I think is interesting is that many of us nerds have probably innocuously fantasized about having a Star Trek-like AI assistant with us, but now that they're taking the first steps towards that, we're starting to realize that in order for it to do everything for us, it has to know everything about us, too.

Nobody was thinking about "the cloud" back in those days. Back then, your data, you programs all lived and ran on your own computer in your home. Most people didn't go online, and if you did, it was mostly to read and download data to use locally on your own computer. Connections were intermittent and slow. The idea that your own data would be stored online was almost unimaginable; even using network-depending applications like usenet or email involved downloading everything first before using it. Online applications were hardly even dreamed of.

Our expectations of how "Star Trek AI" would actually be implemented were completely different than how highly connected cloud-based services like Google Assistant work today.

Anyway, the point being, if the assistant lived entirely in your own computer, it would entirely different. Most people are not concerned about what their "computer" knows about them, they're concerned about what companies and their employees do.

> Our expectations of how "Star Trek AI" would actually be implemented were completely different than how highly connected cloud-based services like Google Assistant work today.

The trope-namer (Star Trek AI) was a ship-wide AI - when considering the ship sizes, it definitely is closer to the "cloud" model and not limited to a private instances on officers' bunker/bridge terminals/tricorders. Perhaps a hardcore Trekkie could answer this question: is there any canon that defines the AIs scope? Is restricted to just one ship, or could it possible be a Federation-wide presence with a presence/instances on ships?

I feel your message is the most important in this thread because it's the crux of the whole concern about privacy and the cloud.

Where technology has failed us most is in the utterly stagnant evolution and maturation of secure private networks. The following is a utopian notion, but had private networks seen as much R&D as the public clouds, they would be significantly less cumbersome than today's clunky VPNs. Imagine all of your devices collaborate directly with one another and with you on your own secure private network—no central cloud servers needed. Your personal assistant is software running on a computer you own rather than a third-party's centralized server.

I still feel this ideal will eventually be realized, but for the time being, no large technology company is willing to take the necessary risks to buck the trend of centralization.

The biggest fiction propped by up centralization and cloud proponents is that it would be impossible to provide the kind of utility seen in Cortana, Siri, Google Assistant, Alexa, et al without a big public cloud. A modern desktop computer has ample computational capability to convert voice to text, parse various phrases, manage a calendar, and look up restaurants on Yelp. Absolutely nothing the public clouds provide strikes me as something my own computer would struggle to do (to be clear, I would expect a local agent would be able to reach out to third-party sites such as Yelp or Amazon at your command in order to execute your desires, but they would do so directly, not via an intermediary).

A few years back, when Microsoft was at the beginning of its Nadella renaissance, I had hoped it would be the first technology titan to disintermediate the cloud and make approachable and easily-managed personal private networks a thing. Microsoft's legacy of focusing on desktop computers would have made it well-situated to reaffirm your home computer as an important fixture in your multi-device life. They could have co-opted Sun's old bad tagline: "Your network is your computer." But they elected to just follow the now-conventional public cloud model, reducing everyone's quite-powerful home computer to yet another terminal of centralized cloud services. Disappointing, but I think it is ultimately their loss. I suspect a lot of money is on the table for someone to realize a coherent easy-to-use multi-device private network model that respects consumer privacy by executing its principal computation within the network.

> The idea that your own data would be stored online was almost unimaginable

Except that's what I did for many years using a computer only as a terminal for an AIX mainframe. My mail was there, I browsed what was the web, used gopher, wrote programs, all stored there.

On top of that, the cloud we have now is commercialized, opaque and constantly under pressure to comply with a government that many distrust, for what I would say very good reasons.

I would like to say that the cloud we have is a privacy concern because we don't know the full scope of data collected, nor what happens to it, nor do we own any of "our" data once it's in the cloud. But not every cloud would have to be that.

There's a perfect world where one wouldn't have to be paranoid about this stuff, but it's not what we have right now.

This is a great point, people didn't imagine a world where we became the product.

Nailed it.

I think there's an additional nuance, that of Google knowing everything about us. If I hacked together my own home automation AI system it would need to know everything about me too and that worries me far less.

...until it's broken into.

I suspect that chances of would-be burglars or identity thieves breaking into Google data are pretty slim, in comparison to a home-installed system.

OTOH both Google and a private person can be strong-armed by a court order, or even a three-letter agency, to open up their AI knowledge vaults.

only if you hacked together your own search, maps, calendar, etc. as well.

Maybe someday that will be a realistic endeavor, but it would take a lot of effort to set up and maintain your own personal versions of all of google's services, and integrate them

The only thing we got wrong. We expected the machines that knew everything about us to be ours. Not someone elses.

Well the missing part is the dedication to ideals and to the greater good of all life that was supposed to be core to the federation. I realize Star Trek is fantasy, but the reason people are more at ease with the omnipresence of technology in Star Trek is because you see people living by these humanitarian and noble ideals. People fight and die to defend thems. The right to self and privacy and protection is held very high in the st universe, even if challenged.

When's the last time Google risked itself or business or any tech ceo risked their livelihood for the sake of the greater good? The problem isn't necessarily the knowing everything part, it's who does what with it that's the problem. I can't really think of any company or person with influence in tech that'd be willing to dive onto that bombshell to protect us all.

> When's the last time Google risked itself or business or any tech ceo risked their livelihood for the sake of the greater good?

February of 2001.

https://en.wikipedia.org/wiki/Joseph_Nacchio

The former CEO of Qwest, a massive telco, spent years in prison for insider trading. He says this is because he resisted the NSA's demands to tap Qwest's network and hand over customer data.

The big difference is the Star Trek computer wasn't using its data about Kirk to provide him "enhanced advertising experiences", there wasn't a big corporation controlling the computer and no government was accessing the computer's information.

A truly user-aligned AI assistant would be great. Ideally in the future these things will not be tied to indirect business models, but rather will be something you buy and all data/services will be under your control.

Capitan Kirk was a government employee. It's implied the government/StarFleet could access the ships logs.

In the Star Trek world they had no advertising because they were a communist society. Everyone dressed the same or slightly differently based on rank. It's interesting how the new movies play over that.

In Star Trek you couldn't choose your AI. In our world you can. At the start of their development most of them are targeted at selling you stuff - but the industry is young and who knows where it will go.

Yes. Also running the AI on local computer made sense because there is no incentive to run it on cloud. Since nothing is gained. Star Trek is set on post scarcity economic society. We will get there once basic income becomes the norm in all countries.

Has Star Trek even a currency? It always feels like a socialist/communist thing to me.

You cannot compare the world's biggest seller of advertisement space with the ST universe. The motivation's aren't aligned: Google/Alphabet want to make sales based on my information.

I agree that I found these oh so clever AI fantasies interesting in my youth, still do to a degree. But I always pictured the data being held inaccessible to humans in general ("Where's my wife right now?") and not in the hands of a golden few with no oversight.

Star Trek's world seems to be a 'utopia' of scientific-military governance. Most of the key players have a military rank, wear color coded uniforms, and appear to be under 24/7 surveillance (which is OK since this is a very nice and progressive scientific-military goverment and you know War On <scary-alien-specie> and all that. :)

In the original show there wasn't a currency but in order to have aliens who exhibit avarice and the worst part of capitalism they had to include a currency (latinum, I think it was called).

Nope, it's a post-scarcity economy.

It doesn't though, does it? The only reason this is a problem is that Google's business is still advertising and they act like our problems can be solved by tools made to sell more ads. The moment I could buy an AI service for like $10 a month (it had to be good), I'd trust them with using my data responsibly.

The Star Trek fantasy is, "Computer, what were the principal historical events on the planet Earth in the year 1987?", and it could totally answer that without sending your entire fucking message history to google for deep AI inspection.

> The Star Trek fantasy is, "Computer, what were the principal historical events on the planet Earth in the year 1987?", and it could totally answer that without sending your entire fucking message history to google for deep AI inspection.

That's part of the Star Trek fantasy. But so is, "Computer, locate Commander Riker" and "Computer, use personal logs and personality profiles from compiled databases to create a personality simulation of Dr. Leah Brahms."

I think people also forget that the Star Trek AI was in a semi-militarized scenario where efficiency and information greatly outweighed individual privacy needs.

I think most fantasies are okay with the anthropomorphic AI assistant knowing everything about us, but don't involve the AI transmitting all of it's data back to "the cloud" where advertisers can mine this data or the NSA could listen in with a secret gag ordered wiretap. Probably wishful thinking, but maybe one day a privacy first company will dip their toes into this arena.

We always assumed they wouldn't spill their beans to third parties. I guess that was assuming too much.

Google doesn't "spill their beans" to third parties-- what Google actually sells is the opportunity for third parties to be included in the advertisements Google is targeting to their users.

Google has a strong incentive to not allow their aggregated user data to leave Google-- the behavioral data Google collects is the reason why Google is valuable; if they start shipping that data off to third parties, suddenly the third parties don't need Google anymore.

(Same with Facebook-- they're not "selling" your data; they're selling the opportunity to target you based on your data, but the data itself is too valuable to Facebook to sell.)

It really is the big problem at the moment with the cutting edge of AI.

ML relies on large data-sets and if anyone tried to release a personal device it simply wouldn't even work, let alone compete with the mass surveillance google/ms/amazon are bringing to bear.

Unless the state-of-the-art in AI suddenly morphs, we seem to be stuck between giving up our privacy or having vaguely intelligent AI.

I personally fall heavily on the privacy side of stuff, but I can see the intellectual and commercial appeal of pretending it doesn't matter in order to get there.

I think we just envisioned a highly anthropomorphized AI: essentially, a very smart and entirely obedient person to serve as the perfect aide. The Star Trek dream emerged before computing technology was very far advanced, and well before the idea of constantly mobile wireless communication. We thought our AIs would be small and physical, easy for a single person to entirely own and unable to remember more details than a human; instead we got unfamiliar algorithms run on machines far away.

The fundamental issue comes down to one of trust and the real question is, do we trust Google to do the right ethical and moral things with our data that they are collecting en masse?

I think some people would be okay with that if:

1. That information were secret (you and service/device/implant) but not a whole company and its third party interests.

2. It wasn't making money for a third party after the initial purchase price for the device, service, etc.

So Siri.

Until it started happening I always assumed it'd be powered by a central computer I had in a clean area of the attic or something. Not some DC somewhere.

"Knowing" in some sense everything about us is not the same as owning that information or trading in it. There are many other possible approaches to applying ML to our personal needs and data, so it is worth being careful about not conflating issues with a particular implementation and issues with the area as a whole.

> in order for it to do everything for us, it has to know everything about us, too

But does it? Does it have to know your birthday? (leave alone the fact that bdays are somehow part of a superkey for your identity).

Why should it know my residence, my spouse, or my CC# (with Apple's TouchID maybe it won't need to)?

Google's concept of AI is too creepy for me. It can be useful without being creepy. They're not even trying to make it less creepy.

Overlay on this the subtext that NSA and other tla's are monitoring all this (leave alone other countries). While I may trust Google, I don't trust them to not be forced to collude with the government.

Not to sound glib, but the idea of persistent data acquisition and aggregation has been pretty well known to be in the path for anyone seriously researching AGI or other human level AI systems.

How could it be otherwise?

I have to admit this is true -- I used to dream of a Star Trek like computer where you can just speak to it but I never imagined that such as system would be strife with privacy and security issues.

You could have realized that from watching Star Trek, as the computer in the Enterprise can always tell the captain where every crew member is, whether there are strangers on board, etc.

Yep. I see us somewhat conflicted about this. We would love the (imagined) benefits, but we have no idea yet how to avoid the dangers.

Bingo. The movie Her was awesome. But rewatch it. In every scene the AI does something cool, think what permissions it would need and what data it would have to have access to about you to accomplish the task. It gets scary pretty quickly.

Daydream is way better.

So, at the risk of making myself ridiculous and branded a Luddite:

I've totally passed on the 'mobile revolution', I do have a cell phone but I use it to make calls and to be reachable.

This already leaks more data about me and my activities than I'm strictly speaking comfortable with.

So far this has not hindered me much, I know how to use a map, have a 'regular' navigation device for my car, read my email when I'm behind my computer and in general get through life just fine without having access 24x7 to email and the web. Maybe I spend a few more seconds planning my evening or a trip but on the whole I don't feel like I'm missing out on anything.

To have the 'snitch in my pocket' blab to google (or any other provider) about my every move feels like it just isn't worth it to me. Oh and my 'crappy dumb phone' gets 5 days of battery life to boot. I'll definitely miss it when it finally dies, I should probably stock up on a couple for the long term.

I'm not sure how much longer I'll continue with the mobile revolution. Pretty much everything I've seen so far that's being branded as AI and the future of mobile computing is just something that saves you from opening up an app. Instead of opening up google maps and searching for directions home, the directions now sometimes appears automatically. Instead of searching for a near-bye restaurant, one is displayed for you. I don't need to enter my flights in my calendar anymore. This isn't nearly as drastic of a change as the original innovations allowed by smartphones. I'm not sure It's worth the trade off anymore.

I'm even more Luddite than you are. I don't have a phone at all (landline or otherwise). You wanna reach me, you email me. The people in my life that care about me have come to accept this. For other things, I read paper maps, plan appointments ahead of time[1], memorise routes, and look up stuff online from my laptop when I find a place to sit down and wifi.

Reading stories like this makes me want to carry a personal tracking device even less.

---

[1] People tend to have fewer emergency reasons to cancel when they can't reach you 5 minutes before the appointment.

Can you mail 911 where you live? If not, for that reason alone I would carry a phone.

People always say that. What about the real emergencies? There's a lot of anxiety and dread that carrying a phone seems to alleviate.

I live in a big city. There's always a phone nearby, including landlines and payphones (they're still there precisely for emergency reasons). There are also passerbys who can help me. The risk of being all alone, having an urgent need to call 911, and be unable to do so is much too small to warrant carrying a phone around at all times.

You can have a cheap, nonactivated phone for calling 911. In emergency mode you do not need a service activated.

Hate to be the bearer of bad news, but your "crappy dumb phone" is already telling someone about your every move with a degree of accuracy ranging from an area around the closest cell tower to within a few feet.

The courts are still deciding when/whether that information requires a warrant.

Yes, I'm well aware of that. That's how Dudayev got himself killed by a missile.

But short of anybody wanting to aim a missile at me I figure that I'm better off with the courts in my country where such information does require a warrant at present (and without any indication that this will change), and without the company controlling those assets trying to 'mine' my profile in order to advertise to me more efficiently.

Dudayev was using a satellite phone, detected by reconnaissance aircraft. But it could happen will cell tower location, too.

Not really a Luddite, I'd think you're more of a Pragmatist when it comes to the Personal vs. Espoused benefits of certain devices or whatnot.

A close friend is a longtime professional software developer. Always interested in mobile. We used to have extensive discussions bout why I preferred carrying a small flip-top notepad and a pen vs a phone or tablet or whatever with a stylus (many have come and gone over the years). In the use-case scenarios I put forward (small lists, secure disposal, privacy, 'battery life') my little notebook frequently was the best approach for me. He disagreed, but that was the point of chatting about our views.

It is a question of how much new technology can add to your life, rather than how much old technology hinders you. Your old phone will function as before. It has the same benefits as before. The new stuff is not going to change that.

The big change is that the new stuff offers the ability to do things in a more efficient way. While it seems to offer very little benefit for individual tasks, some people will see a dramatic benefit while using it for the multitude of tasks that clutter their life. Other people will benefit simply because it enables them to do things that they would not have done before.

None of this is meant to dismiss your points. Personally, I find all of this data mining creepy even when I am confident that they are collecting the data for my benefit and that they won't use the data to my detriment when they are using it for their own benefit. Yet many people don't share that world view. Those people will benefit from Google's services, while nothing is being introduced to hinder the lives of those who don't use those services.

Same here. Have one of those old Nokias and carry it only when I need to be reachable. Cant remember when this has been a hindrance.

I feel like an old fool fighting against its time, but to me all those new applicances are scary not because of privacy (have my data, I couldn't care less), but because of how they shape our world.

Most of the coolest memories I have were the product of something spontaneous, or mistakes, that become close to impossible with a computer and internet in your pocket 24/7.

Assessing what's around you, talking to strangers, actively looking for something without it instantly popping in suggestions after you've typed 4 characters, all those things have been a great source of circumstance-based, little everyday life adventures.

This is the difference between risking buying a random book, or browsing reviews and picking a 5 star one to download.

This is the difference between discovering a place you'd never thought existed while waiting for someone and poking your nose around, instead of standing there, frantically watching their dot on the map get closer to you.

This is the difference between the mesmerizing feeling of playing the first expansions of world of warcraft, versus the tiring experience of the super streamlined versions that followed. Yes, they are less frustrating, but they don't bring tear to your eyes when you thing about them, they just feel averagely satisfying.

A few minutes ago I got up to open the door for my cat, and in a few minutes she'll be back and I'll be interrupted again. I feel like those interruptions are precious. They keep you connected to reality. I could install an RFID cat door, hell I could make a voice activated one in a couple weekends, and I would not be annoyed anymore. I would also never have seen all the things I witness every time I get to that damn door.

If the twilight zone taught me anything, it's that humanity will always have a rebel. If you make life so safe and easy that free will is no longer necessary someone will demand free will.

We might get to the point where society is so safe and easy that insurance company start selling "Risks and Uncertainty" packages.

That reminds me of the TNG episode where two sides of a war long ago switched to simulating attacks. When there's a strike the casualties are required to report to termination "centers" to comply with the simulation.

For consumers this will be a choice between keeping their data private and having intelligent systems that perform better.

So far I haven't seen much, but based on my limited experience I believe customers are going to continue handing over their data to Google and Facebook in exchange for personalised services.

The truth is, the only times my smartphone has actually felt smart is when Google has been mining my information from various services (mainly Gmail and Calendar) and presented it to me at correct time, enhanced with other information they have gathered from web.

I don't think there will be any major backslash from consumers. The old comparison about boiling frog applies here.

There 50 million domestic workers in the world: living, breathing, naturally intelligence, autonomous human beings that people welcome into their homes. Not surprising, given that during almost the entirety of our evolutionary experience everyone one knew knew everything about one. The notion that people would object to a company knowing some things about one in order to place slightly more relevant ads is silly. It just doesn't seem that way in a message board echochamber because their are rewards for self righteous indignation, and not for making the commonplace observation that people willingly make tradeoffs without being victims of "false consciousness".

Almost everybody screens their cleaners and babysitters in some way. Either they are connected through well known friends and family, or for the rich they get checked out for criminality/dangerousness by professional services.

Even then, we have 'nanny-cam'.

Except afaik, those workers aren't coordinated and reporting back to some central agency. If a housekeeper in New York happens to be snooping on their client, it doesn't mean one in Los Angeles is acting the same way. The damage is limited.

Sadly, I think you are right. I'm not sure what it would take to get more people to really care about privacy when they can have convenience instead. The Snowden leaks didn't do it in the US, even though it showed a government that was willing to break the rule of law to collect data on its citizens.

It's simple, regular people are going to get picked off by hackers, lawyers and other predators until much like the steadily rising smog around cities in the industrial revolution, we realize 'oh shit, this is an actual problem because the wind doesn't always blow this shit away'.

Meanwhile actual geeks and hackers will be fine, because we'll have used our intuitions about these things to choose privacy conscious alternatives to mainstream technology.

In addition to which it is increasingly the case that 'privacy' is regarded as an elite thing, and thus will ultimately be sought after by less educated classes. Like how green lawns used to be for the rich to show off that they didn't need to grow crops to survive and now everybody has them and doesn't know why.

Remember Hillary Clinton and the emails. Remember Colin Powell and 'why can't I use my pda in this highly secure area'. These people are the dinosaurs, and in the business world if you're not hack-resistant you're going to go bust.

tldr;

> I'm not sure what it would take to get more people to really care

Their interests get attacked or violated. That is what.

Exactly, there's a choice. "Not OK google" for the author, but nobody's forcing him to purchase the the "creepy" AI and install in his house...

I have an open ended question-- mostly born out of ignorance; But why is this a bad thing? Isn't an artificial assistant that not only knows and understands us but anticipates our needs incredibly useful? In the process, sure they'll collect your info for better advertising, but short of Totalitarian Surveillance or Data Breach Concerns (The former is a bit of a reach if you live in the west, and they can survey you anyway if they really want to, the latter also seems somewhat unlikely)-- whats the issue here? Genuinely asking because I'm trying to understand.

> But why is this a bad thing?

Do you trust every single person at Google, and every single person at every third party company Google shares your data with, now and in perpetuity, to never abuse the data collected on you? Personally, my circle of trust is not that large.

Totalitarian Surveillance is here. In the west. Secure document releases aside, it's too easy to do to imagine a state actor not doing it.

Data breaches of differing severities occur every day, at nearly every company. I would have thought Yahoo was big enough and smart enough to avoid it; but no. Not Yahoo, not Sony, not security contractors, not credit bureaus, not Apple (a'la celebrity photo leaks), not Google (stories abound of individual GMail accounts being hacked).

>Do you trust every single person at Google, and every single person at every third party company Google shares your data with, now and in perpetuity, to never abuse the data collected on you? Personally, my circle of trust is not that large.

(Have worked at google in the past, may in the future, am not currently). You say this as though anyone at Google (or Microsoft or whatever) can go in and search for 'falcolas' and look through your GPS history.

I'm honestly not sure if there is a single individual at the company who had that power. I honestly think that the best thing Google could to is publicize their internal training and documents on personal information, because the regulations and such made me a lot more comfortable with giving Google the sort of amorphous entity my data, because no person is going to be looking at that data.

>, not Google (stories abound of individual GMail accounts being hacked).

One of these is not like the others, unless you're talking about something I'm not aware of. Hacking an individual GMail account requires guessing/taking someone's password, which is not an attack on Google's infrastructure (Unlike the yahoo, sony, apple, etc. examples), its an attack on a bad password.

I agree with you. To help convince people, I realize that we often imagine benevolent leadership, so it helps to give an example such as, "Imagine if you were a Muslim or illegal and Donald Trump were elected president. What could he do with your data?" E.g. find you, search your residence based on your purchasing and travel habits and send you home.

E.g. Wakes up at 5:30 am, travels to a construction site, lives in a house with a large number of people -> signals possible immigrant. Or this:

Detecting Islamic Calendar Effects on U.S. Meat Consumption: Is the Muslim Population Larger than Widely Assumed?

https://mpra.ub.uni-muenchen.de/41554/

We have to think about data not just in terms of our relative safety, but in terms of what could happen in adverse circumstances. And not even just in terms of our own government, but foreign governments.

Sure, there are some trust issues, but just regarding your two first points:

A very limited number of Google employees have access to private user data (only when it's vital to their work) and they have strict policies in place (data does not leave the data centers etc.). Which third parties are you referring to? As far as I know, Google does not give their users' private data to a third party.

> Do you trust every single person at Google, and every single person at every third party company Google shares your data with, now and in perpetuity, to never abuse the data collected on you?

You forgot: every single state which Google is subject to.

There are three levels of discomfort some people feel with this situation:

1. Concern that a single, third-party entity (Google, in this case) might peer into every aspect of our lives, and/or reverse-engineer an exhaustive catalog of our entire lives, by virtue of data collection.

2. Concern that many consumers will unwittingly opt into such control, unaware of the privacy they're relinquishing, and unable to make informed decisions about the possible applications and consequences of the tradeoff.

3. Concern that the custodian of all this personal data (Google) might use, sell, transmit, or turn over the data in ways we had not anticipated or believed we'd consented to.

Personally speaking, I understand these concerns but also understand the potential upside. I'm not 100% sure where I stand just yet. The aforementioned bullet points are presented without editorial comment; just trying my best to articulate what I believe to be the crux of people's concerns here.

The way you phrased the question sounds to me like "what are the shortcomings, leaving aside these terrible shortcomings?"

Having said that: Google is not in the business of making your life easier, but in the business of selling you ads. The data that Google collects about you is incredibly powerful, allowing them to go from a "simple" manipulation to sell you stuff you wouldn't otherwise buy, to full-scale blackmailing you if they see it fit (not saying that this is happening, but if they wanted to, who would stop them?).

It's putting too much power in the hands of a single, amoral entity (like all corporations). That's not good.

> but if they wanted to, who would stop them?

The law, and the economic interest of all the rich shareholders that care about the company's reputation.

Not an ignorant question, but a reasonable one. I think the skeptics believe that society overall does not yet understand the tradeoff. And I agree with them there. I think on a deeper level, people are creeped out that data paints such a deterministic picture of our lives. Sure, I leave home the same time every day, and I leave work about the same time every day. That Google knows where I live and work without me ever explicitly saying shouldn't be a surprise, but not everyone enjoys seeing how their daily lives are so easily circumscribed, even with just passively-collected data.

I wouldn't mind an artificial assistant.

I don't want a company that employs tens of thousands of people, along with a government in a foreign country, along with all the governments on the data route in between, and their employees, civil servants and assorted snoopers of all shades, to have access to the artificial assistant's communications and thoughts relating to me.

All these organisations are made out of people. People with power are inherently untrustworthy; they need enforcement mechanisms to be kept in line, and enforcement mechanisms need to be activated every now and then to stay in working order. That is, occasional abuses are required to keep abuse in line. The thin blue line wavers like a pendulum: it's how we know it's working.

Part of what I fear is that Totalitarian Surveillance is only a "bit of a reach in the west" because we put such a high value on privacy (and personal liberty) that we're willing to defend it. When that goes away then the "reach" will be far easier.

Edit: Interesting comment in another thread: https://news.ycombinator.com/item?id=12639530

> incredibly useful

Sure it would be useful. Sell the assistant as a locally-installed app that guarantees personal data never leaves the LAN and will sell.

> sure they'll collect your info

Only if you let them. Demand better behavior from their software and business practices.

> Totalitarian Surveillance or Data Breach Concerns

What you seem to be missing is that the concern isn't about today's level of surveillance or today's data breach risk. Data generally persists indefinitely once it makes its way into a database or logfile.

To make a claim that these are low risk requires that at no time in the future will surveillance risk increase or data breaches become more common, ... or that the company will run into financial trouble and need to sell your data, ... or that a breach will be forced by a government (not necessarily your's or Google's), ... or that your data will be aggregated into other databases, increasing the "predictive" power and attack surface, ... or any of the other unknown ways your data could be used in the future.

Humans are already known to be terrible at assessing risk, especially when there is a very large separation between the cause and effect. Smoking today giving you cancer many years later is a traditional example. We already know data breaches happen, well meaning employees make mistakes or succumb to corruption, and external powers such as governments or organized crime occasionally take away your agency. Do you really want to claim that none of these risks will ever happen? Because that's the actual wager you're making when when you use Google's products.

For most of history, most humans lived under tyranny or domination if not outright slavery. It's only been the past few hundred years that this mostly stopped in some places.

Maybe we've turned a corner and will never go back to that. But I don't have confidence yet.

>Isn't an artificial assistant that not only knows and understands us but anticipates our needs incredibly useful?

No, not really. Restaurant recommendations and traffic reports are simply not that hard for me to find on Yelp or Waze myself. The "anticipation" here doesn't really help me in any material way.

Here's why I am afraid of Google. Google could have the best intentions, but its wife NSA that Google occasionally sleeps with doesn't. Everything you say to Google Home could possibly be recorded. Storage and Computing power for google is cheap. They can record everything you say in your home. Their algorithms can connect all sorts of information about you. If trump wants to create the next Muslim holocaust, Google and FB have the perfect information.

This is what Elon means when he says AI is like inviting the devil. We have this algorithm in our mushy brain. Its takes about 20 years to train and lives for about 80 years. Its communication bitrate is pretty low (mostly blabbering through mouth) and doesn't retain much information. Only patterns.

Now imagine this algorithm from the mushy brain is run on a silicon chip, with gigabit bitrate, retains almost everything indefinitely and can learn from entire history of humanity.

That algorithm would just need to deceive us until it was powerful enough to wipe us in one sweep.

Google already manipulates humans psychologically to click on their ads en-masse. Giving them more of your personal data is just feeding the devil.

How does Google manipulate humans to click on their ads?

The story of how Target discovers and targets recently pregnant women is a good example [1].

An interesting quote: “we found out that as long as a pregnant woman thinks she hasn’t been spied on, she’ll use the coupons. She just assumes that everyone else on her block got the same mailer for diapers and cribs. As long as we don’t spook her, it works.”

There's no reason to believe Google isn't doing the same thing. And I strongly suggest reading the original article[2], if only for the first two or three paragraphs.

[1] http://www.forbes.com/sites/kashmirhill/2012/02/16/how-targe...

[2] http://mobile.nytimes.com/2012/02/19/magazine/shopping-habit...

Sure, the ads are targeted specifically for you, that's not manipulation, which would be misleading someone to do something.

I realize there's a grey area though.

"AI" is incredibly overhyped. Most of the features and applications I've seen can be relegated into the "that's neat" category, before they are turned off and never used again.

Google recently started telling me how heavy the traffic is on my commute because they've figured out I do it every day, and when I'm doing it. That's nice, but I don't care. I could already get that information from my car's GPS and seeing how red the roads were.

I wonder how much infrastructure, fancy pants machine learning and effort when it to just creating those useless alerts?

Google, as a problem, has already solved the problem they were created to solve: search the Internet. Now they need to find something for all those twiddling thumbs to do, so we get braindead features that tell me what I already know.

> That's nice, but I don't care. I could already get that information from my car's GPS and seeing how red the roads were.

I guess people have different experiences. Personally, I know how to get home from work, so I don't feel the need to turn on my GPS every time I drive home. So I appreciate getting notified when there's notable variances in drive times, without me having to look for it every day.

How often is it correct? I use Waze infrequently (mostly it's turned off in privacy) but it often gets things wrong (it's better than Apple's maps or Google Maps).

I need to get home on time to pickup kids, but mostly just leave a bit early...

I agree with you both. "turning on GPS" could just mean that while driving one has the view of the roads on to see which one's have traffic, not necessarily getting turn by turn directions to and from work every day.

Imagine you're a something like a muslim in the US, and somoene like Trump is elected 5 years from now. You've been here all your life, you have a job, you pay your taxes, you're just a person who happens to be in the wrong place at the wrong time. Much like a jewish person in Poland in the 1940's. Now even back then, it was not easy to escape persecution... but it was possible. In a google world though, there's nothing preventing a corrupt government, or even a corrupt corporate governence from take over, or leveraging this data to your disadvantage. Perhaps your car recognizes you, and locks you in until police come, perhaps you felt safe enough to go to a bar, and that data was forwarded.

Perhaps an exaggeration, the point is, even if you trust google today. There's no guarantee that data will always be held by the people who are google today. We know for a fact the NSA had access to all google data up until at least the snowdon leaks. To me that's the concern about privacy, you have no idea how it can be used AGAINST you in the future.

Everytime someone responds with "I have nothing to hide" I replied along the same lines as you. But I thought about it a bit more and I think the data they have on you is only a small problem when you look at how technology progressed since WWII. Imagine how much easier it would have been today for the SS to extract jews from their hiding places. We have satellites, helicopters with IR camera's, hell, even radars that see through walls.

> Much like a jewish person in Poland in the 1940's

or a Japanese in the US: https://en.wikipedia.org/wiki/Japanese_American_internment

We tend to think that the nazi's were crazy out-there, but we weren't too far off not that long ago, and we can get back there pretty quickly.

From the article- "In other words, your daily business is Google’s business."

From Google- "Google's mission is to organize the world's information and make it universally accessible and useful."

One thing that drives me mad about Google is how they say "the world's information", then ignore 99.9% of the worlds information, and then expect their consumers to give them a pass and not call them to account for how they privatize user information.

Looking at the information that Google organizes and makes accessible and useful I don't see things like "species extinction", "oceanic water temperature history", or say "dolphin linguistic data", equally represented when compared to "my browsing history", "my location history", "my search history", "an archive of my voice searches", "when I leave or return home via Nest", "who I associate with via Google's communication suite". Google is organizing exactly that data which Google can monetize, which is not the world's data. Not a lot of people want to buy data on deforestation so it's much more difficult to get Google to put resources into that. How many people chew pieces of gum until 100% of the flavor is gone? I'll never know, and Google isn't going to help me, because it isn't a profitable data set.

Simply stated, Google needs to stop acting benevolent and start fessing up to attempting to be omniscient in order to be all knowing about its users, not "the world's data".

http://lmgtfy.com/?q=deforestation

First off let me put this out there, people that send me "lmgtfy" links annoy the f#ck-sh#t out of me. I'm proficient at utilizing search engines, thanks pal.

More directly to the point, I was (clearly) comparing the relative resources Google invests in some data sets vs others. Are you arguing that Google invests comparable resources in this type of data compared to the resources it invests in understanding Google's users' data sets?

Google Scholar, Google Books, and Google Earth data for researchers existed before the Google Assistant was even an idea.

Not sure what your point is, either: do you want to get a notification in the morning saying "try to leave early today, as an accident has caused increased traffic," along with another one saying "remember to save to buy an electric car"?

I'm confused as to why in this thread there's very little contrastive commentary on the different stance taken by Apple and Google about privacy.

Apple has made preserving user privacy a paramount goal, investing in research and technology to achieve it with minimal loss (however much it is) of (intelligent) functionality.

I find that a very strong point for the Cupertino based company.

(edited for legibility)

Well, the core of the different stance is just marketing.

Both Apple and Google comply with federal warrants, etc... That's obvious.

Neither of them have any intention of ever letting a 3rd party access user data, that should hopefully also be obvious. As in, neither company sells your information. It'd be a PR disaster, and in the case of Google it would be a massive loss of revenue as it would undermine their entire ad business.

So the only difference is what data they actually have access to, and it's not actually that different. The big difference is iMessage has end-to-end encryption by default so long as you're talking to another iMessage user. That's sorta it, though, and that gets largely neutered by the fact that the messages are then immediately backed up on iCloud anyway and that end-to-end encryption is lost in the process (otherwise you couldn't restore to a new device). Google now offers that, too, via allo's off-the-record, though. Everything else is pretty much the same between Apple & Google with regards to meaningful privacy.

Apple wants (or wanted?) control for themselves. DRM everything; control the products after sale; wall the garden. They became the thing they hated back in 1984.

After Snowden made Apples collaboration with the government economically untenable, Apple may now be willing to let users have control. They fought the FBI to protect privacy on a transparently political charade. They've built hardware and software key protection into the iPhone.

But this is a change, and maybe it's a lie, or maybe it will change back, or maybe they just won't succeed. I'm not rushing out to buy a Mac Book.

I am sorry, but you still believe apple is safe? I mean id love to believe in that there still is some part of tech left, that will not be the reason for reprisals.

Just one data point: After owning 4 nexus phones and being an 'android fanboy' I just bought an iPhone 7. I've decided that while still not that great, Apple's take on user privacy is much closer aligned with my own. I doubt many other people will make purchasing decisions based on that, but I still think Apple will be 'right' in the long run. I think that the resulting applications from all this data collection will never make it past the gimmicky stage for a lot of people. The general public may not care that much about privacy, but they also don't care that much about nearly everything google displayed. I think apple knows this and it's one of the reasons they can take their stance on privacy.

Notice that, despite not giving it Apple's successful marketing, Google also invests in R&D of differential privacy: https://www.quora.com/Does-Google-use-a-differential-privacy...

The idea behind so called checks and balances in the political arena is now needed in the tech arena - more specifically the megacorp arena.

People say, competition will ultimately take care of it. Yet, there really isn't a serious competitor for Google's search engine. And don't even get me started about social networking with respect to your private lives, where the only player is FB as far as I can see.

People say they don't want the government involved, and often for good reason. But if there is no expectation that these tech giants will self-police when it comes to privacy, and people don't want these organizations to be policed by the government either, then how exactly does this play out? How far is too far before we start demanding more respect for our rights from these organizations?

Another thing to think about: when dealing with tangible goods, the creative destruction of capitalism is somewhat reasonable to justify because it is usually easy to see. How does it work with information? Suppose FB just completely blew it for a few quarters in a row, and starts tottering towards its demise, what happens to the "defensible barrier" called data? Does it belong to FB to do as it sees fit, like the assets of a company about to be liquidated? Or is FB going to "return" it to the people from whom it got it? If some other company now got possession of its assets, including data, what is the expectation around what are reasonable uses for such info? Or, is FB, with its trove of data about every single person who has held government office, now just too big to fail?

And all this can be asked just of the data that FB collects from you directly by asking you to fill it in. What about the stuff that it "infers" behind the scenes? What about the "connections" it adds to its social graph without your permission in order to provide a "local marketplace" which apparently gets rid of the "private information" challenge? [1] Not that Google is any better in this regards, of course.

I think the time has come for some serious thinking about checks and balances in the privacy arena.

[1] https://news.ycombinator.com/item?id=12628808

This is literally the perfect end-game for an advertising company: total awareness of need under the guise of 'optimization' or 'AI enhancement'. They can see what your'e searching for, where you're going, when you run out of Mayo in your GoogleFridgeAppAssistant. What better way to offer ads than EVERY time you have a want? It's an advertising utopia!

If they tag on a "where the heck did I leave x" search engine while building out the technology I'm down for the MayoGoogleFridgeAppAss

Just a couple of thoughts.

Is the market really so bad that Google needs to invade people's privacy to this extent in order to grow?

I bet Google's CEO will not use the products himself. Google is almost behaving like a pusher, promising people comfort at the expense of their livelihood (the chilling effect).

Perhaps this should simply be illegal. If people want a personalized AI assistant, why not train the AI on the user's device? I seriously doubt that it has to know everything about everybody's behavior in order to know some things about the user's behavior.

You really have to wonder who truly needs / wants AI in their lives? It's really just being pushed on. Google should be careful not to make themselves irrelevant.

I've been experimenting spending less time with my devices and it's hard because I'm addicted, but life is more fun when it's being lived and not having to even think about technology, leaving devices of all kinds at home and just sitting in a park is a real luxury.

You just answered your own question. A truly awesome AI assistant would be as minimally present as you prefer. If all you do is talk to AI you don't need to interact with a tablet screen, for instance. If you figure out how to make people productive with that then you may have an idea of the next big thing.

Yes but the way it seems to have panned out is we need to constantly feed the machine?

Thing is, I just don't need AI for everyday things. AI to help solve big engineering, medical problems, great, but to help me schedule my life, not really.

Even when travelling, things like Google maps and translte just isolate and distract me in someways. Asking a local about something is really helpful and you can get more out of the intraction than just directions. It's rare language truly is a barrier I find.

I'm even questioning how much I really need a smart phone, it is mostly just a distraction. I remember actually being socially pressured into owning one when I was 18. Never actually stopped and asked myself if I need one, it's just some thing has become a "must have".

I know what you're saying about AI being out of the way though, and it would be excellent, if however; it was completely my data, it worked for me and not for a third party. For example, kept my data private for me, while paying my bills, that might be nice. But basically it would work to keep me focused on the real world, which could be achieved without tech ?

I might sound a by anti-progress here , but It would be nice to just see the right progress.

Well, for a single data point, myself, most of my family, and most of my friends want AI in their lives.

Hell that's the whole purpose behind devices like Google home and Amazon's Echo. People aren't buying them because they look nice, they are buying them because they are a simple voice-operated AI that can answer question and do things for them.

You might not want or like it, and that's okay, but don't act like nobody is asking for this. People have been begging for it since computers were a thing.

What do you do with it? Like what is a typical use case / interaction ? Do you ask it to put on a movie or tell you the time ?

I totally disagree people wanted all their personal information fed to private corporations in the "cloud" BTW. Completely disagree. It's Orwellian.

I guess, reading through these comments, that I'm the only one who wants this future? Yes, sign me up, google. I'll give you more of my information, if you can take it. Can I wear an implant that measures my heart rate, body fat content, blood pressure, glucose level, and brain activity as well? Because as soon as I can I will be the first in line for it.

What some of you don't seem to realize, (and this happens in EVERY SINGLE ONE of these threads) is that:

1) AI is not magic. Yes, we call it "AI", but you use words like "know" as if there is a conscious entity that "knows" something about you. The AI doesn't "know" anything. It's a computer.

2) Yes, actually you can opt out if you want to. Get a flip phone, don't use google services, use an adblocker, block javascripts that you don't like, don't send emails to gmail addresses, etc. Just don't use their services if you don't want them. Yeah, this might be harder. It might feel like you are living in the 1990s/1980s, but it sounds like that is what some of you want.

I, however, want a future where an AI can tell me things like "Flights to Shenzhen are really cheap right now, and you have the discretionary income to afford a trip there. Here is a possible itinerary for you based on the types of things I know you are interested in. You could leave this Saturday and there is nothing on your calendar that you need to be at for the week."

"I noticed that you have been bicycling a lot lately, and based on the patterns of where you go, I think that the following bike trail would be interesting to you. The route is loaded up on your phone already."

The other thing: google is an advertising company. Yes, because I know this, I am able to take this into account when listening to google's suggestions. But here's the thing: I like being [well] advertised to. I have discretionary income, that is WHY I HAVE A JOB. I am going to spend that money on things. If there is an AI that is helping me find the perfect nexus of things I want and things that I can afford, that is a GOOD thing. That is helping me more efficiently spend the money that I got.

Yes this stuff is subtle. Yes this stuff is pervasive. No we don't need yet another "2edgyforme" "if you aren't the customer you're the PRODUCT" articles about google.

I think there are two problems with this suite of crap from Google: the privacy issues and the fact that Google is putting corporate objectives ahead of creating useful things.

It's clear Google wants to "own the home" and all their products were built to further this goal (rather than be useful themselves). This is why Google bought Nest for 12 jillion dollars. And it's why the iWatch failed and Google Glass failed - right now, these are niche products that barely have purpose.

Now this stuff may become integral to our lives, as depicted in so many sci-fi stories, but if they become embedded in our lives and are wholly owned by one huge company, that should be terrifying to everyone.

Here are some real world reasons why: a virus is installed on your Google box through your wifi - now house robbers know everything about your schedule and habits. Your parent goes through your every personal action to make sure you aren't getting in trouble. A spouse uses the system to track your every movement and make sure you aren't cheating. And of course, the gov't has access to all of this data by default. Imagine being a famous celebrity with every action in your house known and accessible to any gov't peon with access and a bit of curiousity. This isn't some conspiracy theory, this is exactly the access Snowden had (and he was a contractor).

It isn't what these products are, it's the direction they represent: complete surveillance of every personal action, stored and owned by one monolithic corporation and the government. And not only is this is sort of where we are heading, it's Google's clearly stated objective.

It reminds me of the 50s when plastics were going to revolutionize everything... which they did, but we melted off the ozone layer before realizing the consequences of slapping new technology across the world. Especially when the benefits are so minimal and the threats are so real - imagine McCarthy with the type of access and control these devices would provide if Google succeeds in pushing this across 80% of homes.

When IR remotes hit college campuses, the game was to shut off someone else's TV through an open door. There's even a one button remote from that era that shuts off any TV in sight [0]. Voice control is like IR on steroids.

Guest at house party: "Ok google, show naked pictures of [host's ex-girlfriend]"

[0] https://www.tvbgone.com

It happens. I've been rickrolled by an acquaintance yelling "Alexa play never gonna give you up by rick astley" thru a screen window. Its gonna be a widespread cultural thing once Hollywood inevitably uses it in a movie or TV show as a joke. I don't watch that stuff, maybe that's where he got the idea it would be "funny" to team up with Alexa to rickroll me.

I find privacy anxiety to be much like electric car range anxiety. Once you have the product its not an issue, it drops to zero, but debate on the internet is extremely hot and heavy right before widespread adoption kicks off. Enormous amounts of toxic anxiety and paranoia bleeding out all over stuff that in practice after deployment just doesn't matter. In other online venues I've been worried about causing heart attacks by suggesting my next car will be electric, and this topic is about the same here.

“Hey Siri, unlock the front door.”

https://www.reddit.com/r/technology/comments/532gmg/my_neigh...

Does anyone else find the tone of this article off-putting? I mean, I agree with the author, but the presentation feels like fear-mongering. Maybe this is what we need to get people to pay attention to the details, but I instinctively mistrust things I perceive as trying to appeal to by fear at a base level, and this triggers that fairly heavily.

I have very conflicted feelings about this article.

It reads very heavy on the tinfoil-hattery. Particularly with things like Google Home's mute button. The single advertised feature of the device is that it's always listening and the author is somehow shocked or confused by the fact that you have to push a button to have it not listen? So much so that they thought it was worth cramming in some stupid sounding snide remark about "true intentions"? That tone immediately undermines the entire rest of the piece.

Many readers are skeptical about the usefulness of personal AI assistants. This reminds me of what Jeff Bezos said about disruptive technologies [1], which I think resonates well among many tech company executives. You (they) need to be willing to be doubted for a very long time.

[1] http://www.geekwire.com/2011/amazons-bezos-innovation/

Any time you do something big, that’s disruptive — Kindle, AWS — there will be critics. And there will be at least two kinds of critics. There will be well-meaning critics who genuinely misunderstand what you are doing or genuinely have a different opinion. And there will be the self-interested critics that have a vested interest in not liking what you are doing and they will have reason to misunderstand. And you have to be willing to ignore both types of critics. You listen to them, because you want to see, always testing, is it possible they are right?

The reaction to this seems awkwardly negative when contrasted with the praise that gushed for Amazon's Alexa products. I am having a hard time tracking why folks seem to feel so differently about google and Amazon having similar access to personal information.

People have a very different perception of their relationship with Amazon (and, similarly, Apple) and their relationship with Google.

Amazon sells you stuff in exchange for money.* This is a clearly understood type of transaction that people understand.

Google gives you stuff in exchange for being able to "sell your eyeballs" to third parties. And, most people actually believe that Google sells your data to those third parties, even though that's not the case, which gets at the fact that this model is not as well-understood.

As far as hardware design:

Echo provides a clear and prominently-placed button (right on top), with an LED light indicating when it's muting the microphone; when the button is active, the entire indicator ring also lights up. This button has equal prominence with the button that can be used to manually activate Alexa. The existence of this button was highlighted when Amazon introduced Echo.

Google Home places the button on the back side of the device, when there is clearly a "front," as defined by the tilt of the top surface. The existence of this button was not highlighted in the introduction at I/O.

* Yes, Amazon also runs an advertising network. Most people don't really know this. And it's a very, very small part of their business.

Amazon sells you things. Google sells your information indirectly to other people who want to sell you things. There's a lot more cultural acceptance around the first business model because more people interact directly with those who do it.

I'm not worried what the engineers who built this will know about me. I'm not worried that Google will centralize this data so if someone hacked it and could drill through the data they could find me.

I'm worried about who Google wants to sell this information to and what they want to do with it. I'm worried about Google working with intelligence agencies to try and target me politically, feed me propaganda, or put me on some list of undesirables.

We can have an ultra-smart AI that does everything for me without worrying about these things. I don't want to pay with my personal information, I want to pay with money. I want Google to stay out of my life.

I am more worried about subpoena on steroids than individual hackers. Big budget crackers are a different thing.

> I'm worried about Google working with intelligence agencies to try and target me politically, feed me propaganda, or put me on some list of undesirables.

This.

If 100 people do A after doing B and you are the 101st who does not want to do A, is it your fault or that of the algorithm? This is no "artificial intelligence" it is a fitness, a mutation and an evolution function, without any true randomness or "chaos" prediction (predicting the future...), but with added Advertising hidden between your reccomendations. Like they can not build flying cars so they reinvent the hovercraft. Aimed at the Internet of things, which will be the biggest tech bubble known to mankind. Sorry i have to vent that somewhere, but all my communication platforms have already been shut down.

I am not trying to inject my opinion. I am looking for genuine discussion. The idea of privacy has always been kind of vague to me. Where do we draw the line of acceptable information sharing and not? If a company is collecting data on our behavior and changing their practices accordingly to maximize their profits, is that immoral? Isn't that essentially A/B testing? The practice of using that data in a potentially manipulative way might be immoral, but is the best way to prevent that really to completely not share anything?

There is no line saying how much is ok. "Privacy" as a concept is the embodiment of personal choice. And everyone may choose differently, in different situations. The only "line" there is, is about whether that choice is voluntary and informed.

It wont be used for A/B testing only. Governments tend to get data and oppress people based on it. Where it comes from rarely matters on that level. You may live in a country that right now doesn't do it, but historically, and currently in other parts of the world, it's a very ongoing theme.

So you can not say 'world hunger doesn't exist because I just had a large sandwich', the issue is more global.

We are just not that evolved yet, and if near 100% aren't where you expect them to be, then you can't assume anything else.

It's ok Google.

Television was ok, too. I used to watch. All the time. TV was the glue that kept us together. Now it's the acid that tears us apart. I no longer use a television.

Google. I love your maps. Your directions. Your free storage. And my earning a living never requires to use you, Google. Just like my TV.

I do expect Google to become something that I no longer desire. Just like the TV. And I think Google won't be able to control or predict it either. Just like tv.

Prerequisite to personal AI worth a damn is data about you worth a damn.

It'd seem windows 10 is setting Microsoft up for this. Google is following suite with its own hardware.

Central task is to infer what you want and help you achieve it, but further, your AI can ask you questions too to work out all sorts of things subtely.

i think eventually, we'll think of I personal information as a commodity or "raw material" and regulate its extraction and trade as such.

My problem isn't with the AI having my data; it's with the company that created the AI having my data. That might be necessary to make an AI assistant practical using current technology, but it's not an inherent requirement.

> infer what you want and help you achieve it

Where is the border between inferring what I want and deciding (for me) what I want? How about an artificially created tilt toward certain consumer or political brands in the process of inferring what I want?

>i think eventually, we'll think of I personal information as a commodity or "raw material" and regulate its extraction and trade as such.

I don't think there is eventuality. We must fight hard now to change our society. We live in a era that determines how future will look like. Politics is path dependent and wrong choices now can have consequences that last centuries.

Economic information asymmetry[1] over consumers and competition benefits few large corporations that can leverage it over customers and competition. They will fight for tooth and nail to prevent consumer regulation. Governments have additional agendas.

1: http://www.nobelprize.org/nobel_prizes/economic-sciences/lau...

It's not just a matter of personal preference. It's well known that mass surveillance is a powerful tool of oppression.

Oppression is not a theoretical idea and not only an historical problem: Government mass-murder in the Philippines, the oppression of Muslims in Europe, of a large religious group in Turkey, of Tartars by Russia (in Ukraine's Crimean province), of so many people in Syria, of populations in all the oppressive countries in the world. The U.S. election could result in oppression of Muslims, Latinos, blacks and others; some U.S. cities already use 'predictive policing' to identify and harass private citizens - what will happen if Muslims becomes an open target? And don't forget anyone who has any interaction with Muslims. Such things have been going on since the dawn of humanity and unfortunately will continue.

The idea that Google and other commercial mass surveillance will not be used for these purposes is a dangerous, irresponsible fantasy; it's lazy, head-in-the-sand thinking, akin to climate change denial: We haven't died yet is the only argument. These systems are not and will not be kept out of government hands: Government already has broad access, as is well known (National Security Letters, NSA spying, Yahoo's recent revelation, etc.). Laws can be made at any time giving government more access, and they will in climate of oppression. Many obtain illicit access, as we know, from the NSA to foreign criminals to antagonistic nation-states. And it assumes that the companies want to deny access; inevitably, some CEO of AllYourDataCorp will support government surveillance and be prejudiced against Muslims or immigrants or blacks. Likely, at least one already is doing it.

IMHO, while it disrupts our plans for IT and wealth, it's absurd to think otherwise.

We are in the mainframe/terminal era of AI right now. Just like with early computing we don't yet have the local resources to do AI locally, so we're using terminals accessing the cloud. The consequence is our data also lives there. It will inevitably change and a personal AI and relevant data will live in your pocket instead (for those that would prefer a limited but more private version).

Well, they came a long way for an advertising company. Most usually die from ad-blockers, but they dodged most of these bullets it seem.

Moving to voice-to-text everything android, just seems as a logical extension to advertise/sell data even more.

Whether is ethical or legal, it doesn't seem to concern them at all when profits are in question.

And that's why I turned Google Now off (but still use some other Google services)

Anyone with a passing acquintance of history and the insideousness of surveillance and will not be blase about privacy or casually trade it in for trivial conveniences that hardly merit the word ai.

You many not 'personally' need privacy or freedom at this point in your life but to casually dismiss it out of hand and fail to consider its import for a functioning democractic society is beyond reckless. Its just one of those things you don't need until till you do.

And thankfully individuals aren't in a position to trade that away unless they can write a new constitution and convince everyone to get on board.

All surveillance does is compromise your society in a fundamental way, and in this case just to add to Google bottom line and ramp up Google's creepiness factor even more. That's a bad deal.

Here's the thing - we're starting to get into territory where we can actually add real value for people again in terms of helping them plan their day, and actually have real AI assistants, computers from Star Trek, what have you. The value here is pretty easy to understand. The problem is that they're all being pushed by advertising companies who make all their money by learning and selling every single bit of information about you.

I want a startup that provides services like this but treats your personal location, correspondence, and behaviors like tax returns and credit card numbers. If we can achieve a good measure of safety and privacy in our messaging apps, we can do it for this sort of data.

If it constantly needs data in order to calculate predictions, then it is no artificial intelligence. It is plain Software, like a calculator. The only reason it is called artificial intelligence is because that would be the only reason to adopt this technology.

Again, it might be neat, having a computer like on star trek, but what if you oppose your government? What if you oppose anything, and suddenly your toaster burns down your house, locks you out, locks you in, reports your every move?

Look at Manning, look at Snowden, look at Assange. They opposed and now they get terrorized by the govrnment and the software they once happily used to use. Look at how i will be treated right here by others.

Stop this Masspsychosis.

Are there any open source/self-hosted projects that do some of what Alexa, Google Assistant, Google Now, Siri do?

Even without the manpower/big data/processing power of a big co I'm sure we could create something that's somewhat useful.

I seem to remember having a lot of the same feelings during the Apple event last month. The Nike+ Apple Watch will "helpfully suggest gear" for your workouts.

Ads have become utterly pervasive, and avoiding using Google's AI isn't going to protect you from them. My Samsung "Smart" TV has ads for Hulu built right into the operating system (despite my being a Hulu subscriber at the time). Windows 10 is basically one big advertisement (at least the consumer edition).

If I have to have ads blasted in my face all the time, I'll take Google's AI-driven ones that at least stand a chance of being less annoying.

Nobody is seeing the obvious benefits here. Wouldn't it be great if we didn't have to really think anymore? Google can just tell me what to eat, what to wear, when to pee, etc. I for one welcome our AI overlords.

People will come to value their privacy more if/when they or their loved ones become victims of harassment, stalking, blackmail, or identity theft as a result of their data being abused, leaked, or stolen.

What I think is really interesting about this conversation is the total lack of conversation. Any and all data collection is either completely harmless (from the corporate narrative) or the end of all liberty and privacy (from the EFF narrative and their leech-like tech rag clickbait headlines, sup techcrunch, you're still the problem!).

There is no concept of even discussing that this might be a tradeoff or a shift in what is perceived as private. There is no consideration given to how we might still do these things that people want while protecting their data. There's no consideration for how people's lives are changed in different ways by this tech.

Nope. It's either a total gain or a total loss.

And that is the real problem here. People are applying their political bad habits to what should be a reasonable and sensitive discussion about the varying levels of tradeoffs we should be willing to give and what the net good we can extract from this technology.

A great example is street view. Street view ultimately has enabled extremely detailed and powerful navigation, complete with a ton of ways to do real time traffic detection. Most people using apps that benefit from this data would say that's a net good, and in general as the tech evolves and traffic distributes more efficiently then urban environments see a similar positive effect.

Of course, the tradeoff is that I can scan a snapshot of your street and if you were there playing football with your kid, walking your dog, or publicly exposing yourself then minus your face I'm going to be able to see all that.

What makes these kind of issues even less clear is that street view enables self-driving car technology (we need the detailed and constantly updated nav systems for them). Self-driving car technology has the potential to totally transform some neighborhoods, has massive potential for assisting disabled people, can completely change the way we ship goods and thusly preserve oil and energy resources for generations to come. But it also has the potential to be a new way for the upper and rich classes of the world to completely cut out service industries and further alienate the economic middle and lower classes.

Why is this meaningful? Because if we don't talk about them then we can't help shape them. If we understand the implications as a society and demand commensurate good from these private industries then it can be an incredible boon to our societies. If we don't, then one of these extremist sides will win and all options for a middle ground where we get benefits and have tradeoffs will be excluded.

That's a terrible outcome.

> ... your daily business is Google’s business.

I can see the day coming where this is their primary marketing slogan.

The scope of Alphabet’s ambition for the Google brand is clear: it wants Google’s information organizing brain to be embedded right at the domestic center — i.e. where it’s all but impossible for consumers not to feed it with a steady stream of highly personal data.

Unless, that is, you never buy any of that junk in the first place -- because like, who needs most, if any of it, anyway? -- and keep going on with your life. Which was humming along just fine before the IoT came along, after all.

God I am loving the justified hatred in this thread :D

I agree that in many respects the current corporate push for the Internet-Of-Things is mainly a wild-west style landgrab for self-serving integration into our daily lives.

However I don't think the IoT has to be this way.

I want all my IoT devices to only communicate to my home gateway, which would run open-source drivers for each device to provide the networking functionality. Problem solved. I don't know why this approach isn't getting more focus!

It is wrong even to ask us permission to let Google and also Apple monitor microphone all the time. If that becomes norm, there has to be hardware switch to disable microphone.

This is the future.

To remain competitive people will adopt new technologies. Google assistant/cloud, self driving cars, CRISPR. Consider what people gain and lose with each new technology, such as the ability to drive, a bio-engineered kill switch, or control their own hardware (windows 10).

All new technologies can be compromised. The ability to process the extreme amounts of data we are generating is already at previously unimaginable levels. Political dissidents or those who interfere with corporate interests can be identified and silenced with false evidence (pedophilia!); media control; and personally targeted DoS of finances, cloud services, etc.

This is the ability to control the world. The corporate world is disincentivized from doing anything about it, and governments don't really get it as evidenced by their hoarding of zero-days [0].

There's a war going on right now. It's terrifying, and awesome. Throw in some global climate change and our next 50 years are going to get interesting.

When the end comes I'll be that crotchety old guy who knows how to DRIVE A CAR and use a general purpose computer.

Hack the planet!

Here is what was possible in 2011: https://news.ycombinator.com/item?id=12528544 [0]https://www.schneier.com/blog/archives/2016/08/the_nsa_is_ho...

I was pissed when Google suggested I take pictures of the restaurant I was in and submit them.

Something to think about -- making the assumption that in the future, a bunch chunk of things currently done with search engines and forms are done by voice command:

Ok Google, order new toilet paper -- order is routed to any ecommerce provider which outbids everyone else to fulfill the order.

Alexa, order new toilet paper -- order copies previous toilet paper order and goes to merchant with lowest advertised price that reports to have that specific product.

Hey Siri, order new toilet paper -- ?

Siri supports a limited version of this today. "Siri, get me a ride" will show which ride apps are available (say, Uber, Lyft and Curb, if you have those installed)

It's interesting you assume that Google will route you to the lowest bidder. When I search for "New toilet paper" on Google right now, the top 6 results are advertisements. Why would they want to anger their customers and allow you ignore them?

Order a new iPaper cartridge from the Apple Store. $39.99.

Imagine the storage implications - i mean they have a hard time to store every click we make with the mouse, now they will have to store every noise and breath we make as well. I see a data store with the size of Canada.

Also how do they plan to make money besides the initial cost of the gadget? Can they push adds while driving? would be too intrusive, or is this supposed to be based on monthly payment, or a tax? Google for government! Wall-E might be needed to clean up the mess after them.

i didn't know that Sting said in 1983 that his song is really a nasty song about surveillance; at least they have an anthem for promotion purposes. http://www.songfacts.com/detail.php?id=548

Now i really don't think that personal assistants are going to be a success. They do descriptive modelling based on what you do, there is no way to evaluate if the suggestions are any good. Without such an evaluation they can't do reinforcement learning. Also they might suck in too much data - that would make it harder to make meaningful suggestions.

I'm always confused by articles bemoaning the AI and tech revolutions, written in magazines that expound new tech revolutions.

I understand Apple and the EFF are staunchly against merging products databases involving the same user's data, but for me this is an essential feature of the google ecosystem. I can ask for traffic and have directions appear on my phone while driving, play movies on the tv that i am looking at instead of my phone, audibly alert me to meetings while at home or work, and turn the lights on and off in the place that I am.

I don't think they are misleading people, the mute button pretty strongly implies the duality that you can't hear and un-hear things post-hoc. In addition they dont hide the fact that you are talking to a computer at a company by obfuscating it with some quasi-futuristically named caricature. As is often with these article "always listening" is far more misleading, an embedded keyword processor is listening for keywords, and only if they match the phrase "ok google" are they sent to google servers. otherwise it just sits there sharing nothing.

I'll note that my iPhone can do all those things but without my data leaving my own hardware (or at least not unencrypted). You don't _have_ to ship all your data offsite for the functions you enjoy. It's just that Google would rather examine all your data in-house, presumably for their own benefit and not for ours.

With faster multi-core CPUs, much more RAM and SSD, smartphones are probably powerful enough to run a privacy respecting AI assistant locally: watch your web access, phone access, location, and do all processing locally with no data leakage. Ideally it would be open source, or at least from a trusted company that made money only from selling the app, and not off of our personal data.

So I see both sides of this privacy/AI debate going on and am wondering, what is the solution? Is it technically feasible to create a useful future AI without compromising privacy? Because if not, I fear for our future. While I agree with what everyone on the privacy side is saying, I believe in the long run the consumer value AI will win, leading us down this path.

The new assistant in Allo which automatically suggests message replies to images by detecting what's in the image... it scares me.

Ok, so no personal assistants and high-tech homes for anyone. Done. There, you happy now? Because if Google isn't being fed all that data, it can't provide those things. You either have it or you don't.

Edit: I've made this point before, but your data is a currency. Spend it wisely (or even not at all). It's up to you.

It's fascinating to think about how exciting this technology is -- passive monitoring by helpful AI that can drastically increase convenience and efficiency.

But we've seen that Google is happy to turn over massive amounts of customer data to government without a warrant and without alerting customers to the practice, which makes the technology seem ominous.

First the GPS, then the microphone, then the camera, accelerometers, 3D touch sensors, etc. Gait, affect, and all sorts of factors will be able to predict criminal behavior before it happens.

Let's hope the next generation of tech giants will take customer privacy and freedom seriously and avoid the dark patterns and privacy violations of the current era.

Only now, when it's likely too late, can we actually get a glimpse of the sort of Orwellian dystopia that so many have warned about in decades past.

The idea behind ZeroDB might be applicable here.

All data generated by a user is encrypted and stored in the cloud with the decryption keys on the user's device. This way, the service provider (eg Google) can't read your data. The major advantage of this approach is that the user is in complete control of the data. The drawback is that service providers and AI systems will be starved of data that enable targeted ads/recommendations.

A good middle ground might be to offer users an option : 1) Give us your data 2) Pay up and we will not collect any data.

Thinking deeply about the state of the internet, I think we have to move towards a model where users pay for services they use if privacy is a concern. As it stands, a lot of services offer free services in exchange for our data which is monetized through ads.

What I'd like to see is some good open source software that can compete with Google Assistant, Siri, and Alexa. It looks like there are some promising projects, but nothing turn-key yet. I'd like to be able to simply apt-get a package, and have voice recognition on my box.

Voice (technically speech) recognition is relatively close to being doable at acceptable quality with open source tools.

The hard parts of the Assistant, Alexa, etc., are basically everything else.

I think it's right for people in the know to be concerned. The direction has all kinds of possible disastrous consequences. But it also has lots of amazing dual-use possibilities for making our lives more fluid, and technology more magical. You can't deny both.

Trying to take some big stand against it I don't believe will work. Look at all those who took a stand against the 2nd Iraq war -- they were drowned out. And now everyone thinks the opposite. Culture/society always pushes a particular direction until there's a very big disastrous reason to think otherwise. Until something clearly really really bad happens this is the track we're on, like it or not.

Reading this brings the thoughts to Kahlil Gibrans book the Profet. The section "On Houses" reads:

...

Or have you only comfort, and the lust for comfort, that stealthy thing that enters the house a guest, and becomes a host, and then a master? Ay, and it becomes a tamer, and with hook and scourge makes puppets of your larger desires. Though its hands are silken, its heart is of iron. It lulls you to sleep only to stand by your bed and jeer at the dignity of the flesh. It makes mock of your sound senses, and lays them in thistledown like fragile vessels. Verily the lust for comfort murders the passion of the soul, and then walks grinning in the funeral.

...

I was really against getting updates about where I usually travel or other notifications that are constantly tracking me. But after a while I really like some of the tracking and notifications. Now I love when google reminds me to leave for a appointment and gives me directions, it really saves.

I do hate being tracked, but I have slowly started to like the connivence of it. Having all my information on Facebook, Linkedin, Instagram, and everything else privacy has really gone down. If my Nexus 5x can save me time, I will sacrifice some privacy.

> We are excited about building a personal Google for everyone, everywhere

Not a problem if it run on an appliance at my home, disconnected from all the other appliances in other people homes. That would be a truly personal Google.

The trade off here is technological advancements in AI for lose of Privacy. Everyone can have their own opinion on this but in reality, consumers are the best for collecting data. The great thing about the grand scheme of it all is that if you don't like the privacy you're giving up, don't use it. There's always going to be the lesser known alternative that doesn't track any of your data. I don't see why articles like this arise as it's clear as day that without tracking and getting analytics, AI won't improve.

From the article: So the actual price for building a “personal Google for everyone, everywhere” would in fact be zero privacy for everyone, everywhere.

That has no basis. It is completely possible to do what they are doing by keeping everyone's individual privacy intact. And if I go by Google's privacy policy, that's exactly what they are doing. And I think it's in their best interest to keep it that way, because the day it comes out in public that our privacy is not safe with them, everyone will stop feeding them more data.

"Zero privacy" here means privacy from Google. Based on your reading of their ToS and your beliefs about their best interest, you may trust them to do no wrong and you may therefore be comfortable with Google knowing everything about your life, but not everyone feels the same.

> It is completely possible to do what they are doing by keeping everyone's individual privacy intact. And if I go by Google's ToS, that's exactly what they are doing.

Could you give examples of how that can be achieved and how the ToS goes about stating that they are doing this?

How much hardware would it take to keep 80% or 99% of the data local?

Internal SSD?

Thus article is rightful in raising concerns for privacy. But it's just repeating and rehashing same things. Google had been devouring data for long time.

I've long since given up hope that I can achieve any sort of privacy. Even were I to do my utmost I will still be on the periphery of others using tech, visible through their actions. The only thing I really want is for the content of my messages to be private. This can be achieved through end-to-end encryption. I push everyone around me to install and use Signal instead of Messenger or Google Talk.

I would love to have personal devices that collect my private data and apply AI for my personal benefit, but only then if all data stays inside the device under my exclusive supervision. As soon as this data is sent to some cloud service beloning to a company in the business of generating wealth and power, I totally lose my appetite.

I'm considering purchasing a Home. I already have an Alexa, which I really like, but based on what I know of Google's data and AI (and their service APIs, which I assume/hope will be extended to Home), I can't imagine it not being significantly better than Alexa.

That said, I agree with the OP's takeaway: people should be asking questions. I mean, people should have been asking these questions long ago, even as just search users questioning how Google manages to return such geospatially relevant results. But most people don't even stop to think about it, as that kind of thing is just taken for granted as the thing computers just do.

Maybe with Google's data and AI in the form of a physical, listening bot (I don't know many people who use OK-Google on their phones) will be the thing that clues people in. I'm mostly comfortable with Google's role in my life (though not comfortable enough to switch to Android just yet), but I'm aware of what it knows about me. If AI is to have a trusted role in our lives and society, people in general need to at least reach the awareness that the OP evinces, if not her skepticism.

One thing that I think is lost in a lot of the comments here is that, to a large extent, privacy is experienced, not factual. That is, in many cases, the breach of privacy is the act of mentioning something that should be private, not whether or not the system (or the person) knows that thing. This is something we tend to intuitively understand in our human relationships, but one that somehow seems to be forgotten in the design of these systems (or, at least, the conversations about them). We need good ways to tell the Google Assistant that something is private (or for it to figure it out for itself) -- even if it still possesses the underlying data.

(There are, of course, situations in which the actual existence or not of specific data is what matters, but I think those are less relevant to the success of something like Google Assistant than the perception of privacy -- and that perception is important, regardless of the underlying data.)

Once we have surrendered our senses and nervous systems to the private manipulation of those who would try to benefit from taking a lease on our eyes and ears and nerves, we don’t really have any rights left. - Marshall McLuhan

I seriously wonder if this article would have a different tone if all the same products and vision were presented at an Apple Keynote with Apple branded devices.

Would it be different? My gut says this article would have had a different title.

https://en.wikipedia.org/wiki/The_Circle_(Eggers_novel)

How's this different from Apple or any other company that wants to stay relevant? AI is the future, like it out not.

Many software companies have been trending toward on-device AI though that largely avoids these privacy issues.

Would you elaborate for me, maybe toss out an example?

Why is everyone so incredibly paranoid?

You can't seriously be asking this question in a post-Snowden world.

Why is everyone so gullible?

Because they have good reason to be suspicious of massive data collection by companies and governments.

Internet Points are to be had.

in WAZE they went ahead and dropped the "while using the app" option for gps privacy. it has become rude actually, so now i turn off location services instead of micro-managing the unmanageable.

I will never put an always on Internet connected microphone run by an advertising company in my house. I don't care if it literally spits out cash and cures cancer.

What about an Android phone?

Same here.

The constraint is going to kill Google and Apple because a future competitor is going to manage to offer service without requiring the data in raw form, an example of which would be Agent Based Computation in say 15 years time. Because of that it shall become the standard. Google and Apple will be stuck because their stakeholders include parties which require the raw form even if Google/Apple did not. That opens up a good line of attack on a previously impenetrable business model.

The legal implications of visiting somebody's house should be considered. Did I sign anything saying Google or Apple could use my data? No. This opens up the door for class action law suits in some US states at least. Unless they are smart enough to be able to dodge this somehow, perhaps with a contextual local filter for new voices and a permission request made of same, but I doubt it, they probably don't have the incentive to look at the ugly.

cat <that article> | sed -e s/data/your data/g

Star Trek or North Korea? In the limit that's the choice.

well my iPhone and my navigation systems both will tell me how far I am from home/work/parents depending on my location all without my asking.

I'm waiting for the barbarians.

That's the big difference between Apple and Google, Apple just want $$, Google want to know everything at a more deeper level.

tl;dr: Google wants your data on its servers.

As opposed to all the other companies out there I guess?

I'm willing to sacrifice an amount of privacy now for future potential improvements to my life.

That's great! I'm willing to sacrifice any amount of your privacy too for future potential improvements to my life.

But there would have to be current and actual improvements to my life before I'll start considering handing out my personal data.

How about for potential risks? What if thieves/criminals get a hold of that data? 'Oh he goes to the pool every day 2-3 and lives alone with no home alarms'.

But why are these the terms of the bargain at all? Why is there no room to negotiate?

People want privacy. They just feel helpless and powerless to stop hemorrhaging personal data.

So because Google makes phones and hardware (like it has been already for quite some time), there's somehow an even greater threat now on our privacy? I don't buy it.

N.B.: I don't work for Google.

I've come to believe that there is nothing that humans can do to resist the privacy encroachments of our machine overlords. Don't blame Google - it's a universal inevitability.

On the contrary, humans have done plenty to resist it.

I'd be completely fine with sharing most of my data with Google, really. Personal and Private interlock a lot with each other, and if Google wants to give me a personal experience they're going to have to use some of my private experience.

Reply on news.ycombinator.com